How to choose max layers and units to search over in hyper parameter tuning
When performing any hyper parameter tuning, let's say random search for simplicity, and I want to search over a minimum to max units/nodes in a layer, and a minimum to max number of layers, are there rules to guide what is a large enough number for my search?
Currently all I know is that should be good enough/large enough, let's search in there. I could be not searching a large enough space, or searching a space that's far too large and doing unnecessary searching.
Is there any way to help decide what space should be searched over, before one begins?