According to historical records, Kuala Selangor had settlements as early as the 16th century. Other areas such as Jeram 10 miles from Permatang, on the other side of Sungai Selangor (Selangor River), also existed. These places were under the rule of the Malacca Empire and after that the Johor Empire.

The history of the fort began with the downfall of the Dutch East India Company after the fourth Dutch-English War (1780-1784). When Dutch influence declined, Selangor, backed by Johor and Riau, invaded Melaka, which was also under Dutch rule, but failed. The Dutch then invaded Selangor and conquered Kuala Selangor in July 1784.

During the invasion, Fort Malawati collapsed but was reconstructed and renamed Fort Altingsburg, after the Dutch Governor-General. One year later, on June 28, 1785, Sultan Ibrahim with the help of his brother, Dato' Penggawa Permatang Mahabijaya (Penggawa Tua) and Bendahara Abdul Majid of Pahang drove the Dutch out in a night raid. The attack, which began after the evening prayers (Isyak), succeeded in securing Bukit Malawati before sunrise. From that date until the British intervention in 1874, this fort was to be the scene of many a battle.