Ngah Ibrahim was a prominent figure and strong resistance fighter against the British colonials. The physical legacy of Ngah Ibrahim is his fort, built in 1854 on two acres of land, in Matang, about 8km from Taiping town.

Within the fort's compound were his official residence and a tin ore store. The fort was by far the strongest Malay fort ever been built in Perak. However, the fort was taken over by the British soon after Ngah Ibrahim's downfall.

Over the years, it has been used for various purposes, among them as the court for the prosecution of Dato' Maharaja Lela and Dato' Sagor charged with the killing of Perak first British Resident, J.W.W. Birch. The duo was found guilty and was hanged to death in front of the house on January 21, 1877.

In 1913, the house was turned into the first Teachers' Training College for Malay teachers, known as the Matang College.  Ten years later, the college was closed for the new Sultan Idris Training College in Tanjung Malim.  Subsequently, the residence was turned into the first National Primary, Sekolah Rendah Kebangsaan Matang. During the Japanese Occupation, the Japanese Army made the fort its headquarters.

Sometime after Independence, the Museum Department took over the fort with the walls and Ngah Ibrahim's resident still intact, and turned it into the Kota Ngah Ibrahim Historical Complex.