Dave makes good points, but theres a lot more than just quality to think about.
In terms of quality my Legra and my Dean custom (since sold) were very similar. Legra took it, but the dean was very well made (albeit from lesser woods, but still very good - the only objective flaw with the guitar was the nut was a little too high: easily fixed).
But I sold the dean because I didnt get on with some things about the design. Subjective matters that dont reflect on dean. The same has been true with various top end guitars I've played; considering only the quality if assembly and finish - jackson soloists (great, but I dont like painted necks), top end RGs (great but I cant be arsed with another floyd-a-like), SOME top end gibsons (dont like the scale or shape enough to shell out that sort of money, do like it enough to have an Epi LP kicking around), top end ESPs (mix and match all of the above). I've also seen some shockingly ropey guitars from Gibson, Fender, Ibanez, Caparison (only one, admitedly) and others (both good and bad), so I think max quality is possible with an off the shelf guitar, but far from certain at any price. Thats not the case with customs (in theory at least)
Which is a huge part of what Custom is about: getting exactly the guitar you want. Which forces me to make a deliniation in custom shops, into Bespoke and Custom:
Bespoke - Legra, Wez, Rek, Feline (?) etc: you get whatever you want.
Custom - You get a guitar built specifically for you, to order (so its custom) and its customisable, but you have to buy into a particular line of guitars that that maker thinks is The Ultimate Guitar (or is a slightly different and marketable idea) - Blackmachine, Organic, Driskill, etc. You dont get whatever you want. You get a custom built and possibly customised version of their line of guitar.
In terms of cost, I think you get much better from Custom and Bespoke guitars, for various reasons, one of which is the lack of name (but then you have Name Customs and Bespoke guitars like Conklin and Alembic, that charge SERIOUS money)
There is a cutoff to guitar quality - how well put together one can be, and many off the shelf gutiars hit it at around £500, or even less. Materials are a different matter, because there are pseudo objective measures (like grade, sg, grain density, oil and moisture content) and subjective (what they sound like when a certain guitar is made from them) and asthetic value. Then you have distinctive peices of exotic woods that are really quite hard to price, or evaluate the value of to any given person - they can be very pretty, very unique, but the complex grain patterns wrecks the structural intetgrity of the woods and makes them tonally useless to boot, so 'quality' is subjective there again.
Long story short, theres a point where a guitar just cant be perceptibly better assembled and superior wood (by any objective measure) cant be found and both top end production, and custom and bespoke guitars hit that point, but not with equal consistency. I think that if you can find what you want in an off the shelf guitar then its entirely possible that that guitar can also max out any possible objective measure of guitar quality, but theres always a chance (whatever chance it may be) that even top end off the shelf guitars from the same factory will be flawed or outright cr@p. Customs and Bespokes are very consistently high quality and you know the materials in them are what they're supposed to be (not like "African mahogany", for example), you know its all going to be made and finished to near as well as a human hand and eye can manage (all things being equal, from reputable and skilled builders like the ones mentioned above), but there the drive isnt high quality, thats just a given, or should be, its getting the exact guitar you want. Apples and oranges.
