Welcome.
I started out brewing 1 gal batches to get my feet wet. Made 3 extract batches then moved to 3 gal batches, now up to 5.
Get the book How to Brew by John Palmer. It's a great read.
I know you said you wanted to start with all grain. I do all grain but BIAB, meaning I mash and boil in my kettle as opposed to having a kettle and mash tun. You will also need a fermenter, temp control, etc.
As for as your equipment, it depends what your plans are. Since you never brewed, I suggest hooking up with someone, like at LA Home Brew to watch them brew. It will allow you to get your feet wet. You can always start with a 1 gal kit from one of the on line retailers, but I would think you will quickly outgrow that.
If I had to do all over again, I would have started brewing 3 or 5 gallons batches. With that, you would need at least a 10 gal pot plus a mash tun. You can check out Spike Brewing or SS Brew Tech for a kettle. There are others out there but those are probably two of the top. You will also need a fermenter, plastic, glass or stainless.
It basically all comes down to what you want to spend. I'm thinking about moving to electric but that will not be until sometime next year.
There are some nice electric systems out there, SS Brew Tech, Spike, High Gravity if you are looking for a turn key set up. If you have the knowledge to build your own controller, go for it. I don't so I would have to go the turn key route.
Good luck with whatever you decide to do.