Using a standard module instead of a development board is a good suggestion because it allows full control over power usage.
A minor disadvantage is that a module is not breadboard friendly so if needed one needs to add a suitable bare bones adapter (one that does not draw power) or design a custom PCB.
Like @LoRaTracker already mentioned you will have to find/use an efficient LDO that can deliver sufficient power for the module. LDO’s for larger currents usually also have higher quescient current which is exactly what you want to minimize for minimal deep-sleep current.
Having a shortlist of suitable LDO’s would be nice.
LiFePO4
When using LiFePO4 cells it is possible to directly power an ESP32 module from the battery, without using an LDO.
Lowering clock speed
When use of WiFi and Bluetooth is not needed, the ESP32’s clock speed can be lowered to as low as 10MHz which minimizes power usage when not in deep sleep. The minimum for WiFi and Bluetooth is 80MHz. See: ESP32 power consumption at different CPU speeds.