If you still want something lighter than XFCE you can try a window manager like dwm. It’s really light and you can customize the entire things by modifying the source code. If you just wanna customize simple things (color, font, key binds, etc.) you can edit a neat little file called config.h
Sucuk