Ok, I found the problem. It is a bad connection somewhere…
If I put pressure on the back in the middle, I get power back. Maybe the battery bend the board a little and something got disconneted, or a copper lane broke.
Thanks everyone for helping debug this problem.
So ‘bad connector’ came most close. 
(‘Poor contact due to mechanical issue’)
I used
#include <axp20x.h>
And then for turning ON
axp.setPowerOutPut(AXP192_LDO2, AXP202_ON);
axp.setPowerOutPut(AXP192_LDO3, AXP202_ON);
axp.setPowerOutPut(AXP192_DCDC1, AXP202_ON);
axp.setPowerOutPut(AXP192_DCDC2, AXP202_ON);
axp.setPowerOutPut(AXP192_DCDC3, AXP202_ON);
axp.setPowerOutPut(AXP192_EXTEN, AXP202_ON);
If you want OFF just change AXP202_ON to AXP202_OFF
LD03 is for the GPS
!! Chip resource table
| CHIP | AXP173 | AXP192 | AXP202 |
| -------- | ---------------- | ---------------- | ---------------- |
| DC1 | 0v7~3v5 /1200mA | 0v7~3v5 /1200mA | X |
| DC2 | 0v7~2v275/1600mA | 0v7~2v275/1600mA | 0v7~2v275/1600mA |
| DC3 | X | 0v7~3v5 /700mA | 0v7~3v5 /1200mA |
| LDO1 | 3v3 /30mA | 3v3 /30mA | 3v3 /30mA |
| LDO2 | 1v8~3v3 /200mA | 1v8~3v3 /200mA | 1v8~3v3 /200mA |
| LDO3 | 1v8~3v3 /200mA | 1v8~3v3 /200mA | 0v7~3v3 /200mA |
| LDO4 | 0v7~3v5 /500mA | X | 1v8~3v3 /200mA |
| LDO5/IO0 | X | 1v8~3v3 /50mA | 1v8~3v3 /50mA |
Maybe not really relevant anymore as it is already an older board.
But last week I found a simple way to overcome the chicken-egg problem with the AXP192 on the T-beam 1.0 when mounting a SSD1306.
The silkscreen labeled 3V3 and 3.3V (why different?) actually are programmed by TTGO/LilyGo to output 1.8V at boot. No idea how yet, but they seem to have changed it from the default according to the datasheet.
As mentioned by others the SSD1306 will clamp the I2C lines to its Vdd level (+0.3V) which is still too low for the ESP32 to work with or at least reliable.
So what I did was, I put a simple 1N4001 from GPIO-0 to the not used “3.3V” pin on the T-beam board.
Since GPIO-0 has to be pulled-up to boot the ESP32, it is a gentle nudge for the display to not pull the I2C lines down to its low power level so you can address the AXP192 chip and set it to output 3.3V.
N.B. I did also make a PR to the ESP32PAXcounter repo, which has now been merged so hopefully others will find it useful: Display & LED - Documentation
I also used the PPK2 to measure its current draw from the battery clamps and in “sleep” mode (ESP can quickly boot and some stuff was still powered) it draws about 1 mA.
When the AXP192 is set to turn everything off (RTC still seems to remain powered) it draws about 50 - 60 uA.
I do think it is quite common for other I2C devices to pull the I2C lines to its Vdd level using diodes.
Quite a lot of microcontrollers do the same as a protection for higher voltages on their GPIO pins.
Maybe it can be used as a rule of thumb that whenever the datasheet mentions “Vdd + 0.3V” as absolute maximum, implying the chip has internal diodes between GPIO pin and Vdd?