Your gearbox has an input shaft and and output shaft connected together by a number of pairs of gears, one pair of which is meshed together at any one time depending on which gear you've selected. When you push the clutch pedal in the input shaft is disconnected from the engine, but the ouput shaft is always connected to the wheels. When you move the lever to select a gear the syncros for that gear have to force the input shaft to speed up or slow down so the two shafts are spinning at the right speed before the pair of gears can engage.
If you are driving along in fifth and you try to change into fourth the input shaft only has to be speeded up a little bit, so it doesn't take a lot of force to push the lever into that gear. But if you try to change into first it has to be speeded up a lot, so its really hard to push the lever into that gear. Its not that you can't. Its that you have to push really hard. Or do it slowly so the input shaft has time to speed up. Or engage neutral, let the clutch out and blip the engine so it speeds up the input shaft, then press the clutch again and select the lower gear.
