18:33:29 # Life Found the reason podman build was failing as normal user. dbus-user-session was missing and podman wasn't able to connect to systemd user session. This resulted in trying to use cgroupfs instead of systemd for managing cgroups and other things also broke.
18:54:05 # Life user namespace not enabled when running podman. Completely forgot about this but seems like I need to enable user namespace for podman. sudo sysctl -w kernel.unprivileged_userns_clone=1. Not quite sure when this started being required.