Babylonians, similar to Sumerians, practiced a polytheistic religion. Their chief god was called Marduk. The babylonians built temples called ziggurats to worship him and other gods. From the epic poem of this religion, Enuma Elish, you can learn more about the base of their religion and how Marduk became the head "Bel", which means lord.
Most records of Babylonian mythology are dated all the way back to 700 B.C.E. In 700 B.C.E. they were transcribed into clay tablets and stored in library of the Assyrian king. However, the two major Babylonian epics probably originated around 2000 B.C.E. They had 7 major gods, Marduk being the chief of all gods.