The history of Covington begins when General "Mad" Anthony Wayne had a fort built near the Stillwater River. The fort was dubbed "Fort Rowdy" in reference the the conduct of the soldiers that were stationed there.Operated by the Covington-Newberry Historical Society, the Fort Rowdy Museum is located at 101 E. Spring Street across from the J. R. Clarke Public Library and the post office. The building itself dates back to about 1850 and has served many tasks for the town, including a schoolhouse. What you will find inside is an impressive mix of Covington/Newberry Township artifacts.

The walls of the downstairs are covered with early photographs that date from the mid 1800s to the mid 1900s. Display cases are full of products and promotional pieces from local merchants and political campaigns. Some of the products represented are, soap, talcum powder, cigars and Rogers drums.

In the back of the first floor are tools and parts of early Newberry Township mills, mixed with saws that were used to cut ice out of local ponds in the era before electric refrigeration. There is also a small collection of flower bags from mills in the Covington area that are long gone. On one wall is a painting of Leonard Covington, an officer that served with General "Mad" Anthony Wayne during the Indian wars, and for whom the village of Covington was named after.

Not to be missed, the upstairs contains a collection of antique toys and clothes, as well as kitchen items once produced in the Covington and Piqua areas. The museum also has a collection of vintage clothing, including military uniforms that were donated by longtime residents.