Matt brought up the point that he didn't think gardens were very good because you need to get at least 4 great person points (or another multiplier) before they'll do anything because the game doesn't display decimal points for great people like it does for things like food, hammers, and science. I did some testing by making a note of how many points I was generating in a game where I have a 75% bonus to great person generation and was making 9 base great musician points. With the 75% multiplier that would be 15.75 points per turn. I went 15 on one turn and then 16 on the next which shows that the game is tracking hidden decimal points when it comes to great person points.
There were other criticisms left in the comments on my post about wanting to get a garden. Dave said he spreads out his guilds (presumably to dampen the impact of having to run so many specialists in one city) and Sky decided that the hammers to build and upkeep cost to maintain of a garden makes it terrible.
I definitely used to spread out my guilds when they first came out. But back then I made a point of not building a granary if it wasn't going to give me a ton of extra food. I also only built trade routes for gold if I built them at all. (I kept having the AI declare war and destroy them all so I ended up just not building caravans and cargo ships after a while.) But it turns out a granary for just 2 food is enough to cover a specialist so it's a lot like a granary could give you 3 culture and 3 great person points! Or it can give you 2 gold and some great merchant points. Or it can just grow your city faster... Granaries are actually awesome and I've really started building them all over the place. And as a side effect to building granaries all over the place you can start transporting your food around. I used to not do this both because I didn't build granaries and because I thought it would cost food in one city to add it to the other. No, it turns out you actually just generate free food! And you should totally do that! And if you've been doing that to one city for most of the game it ends up really big and has no problem coming up with all the specialists you could desire. (If you go tradition that one city should be your capital due to monarchy.)
I definitely think any time you go tradition you should be building all your guilds in one city and that city should be your capital. If the city doesn't have a lot of food to make that viable you should be shipping in extra food from the outside to make it happen. Get the extra gold and happiness out of monarchy, get the extra culture and great people. And get a garden/national epic in that city to make more great people!
The opportunity cost of building a garden could be a thing. Gardens cost 120 hammers and 1 gold per turn in maintenance. Especially early on, 120 hammers is many turns worth of production and you could get something else instead. Especially if you don't have any guilds there's really no reason to be picking up a garden early. The writer's guild itself only costs 100 hammers! So paying more than that amount again to get only 25% of some of the benefit only makes sense if the writers' guild is a fantastic deal. I actually think it is, since great writers can be sacced for a big burst of culture to get a policy, but I can see how others may think otherwise. Once you've spent the 150 hammers for the artists' guild and especially the 200 hammers for a musicians' guild the extra 120 for a garden stops looking like such a big deal. Especially if you have other things going on in the city to generate great people like wonders or different specialists.
The 1 gold upkeep feels like nothing at all, especially if you build all your guilds in one spot. You're not building a garden in every city, so whatever 1 gold. Whatever I say!