Because by the very nature of the camshaft's purpose- to allow air/fuel mixture in to the cylinder- in order for it to do more of that, it affects power down low.
Meaning: As we discussed, the camshaft opens and closes the valves. In order for the camshaft to increase power, we typically order a camshaft that does two things:
It opens the valves further, and for longer. This increases valve overlap (the time during which the intake and exhaust valves are open at the same time), and can in extreme cases also affect vacuum. This results in a rough idle, and a engine that cannot fully utilize the extra air fuel provided until higher in the rpm range.
Lobe separation angle can also affect where the power is made. Wider means power higher in the rpm range, more narrow LSA means better power down low.
Another factor in all this is how a different camshaft affects cylinder pressures. A lumpy cam with a wide LSA will see lower cylinder pressures.... and thus less power down low.
LSA is the distance in degrees that the intake and exhaust lobes are separated on the camshaft. Measured at the peak point of lift.