Some sort of stronghold must have been here since the early Middle Ages. When during the long wars between Holland and Utrecht, count Floris V of Holland conquered this stretch of the river around 1280, he built or enlarged the castle immediately. Thus controlling the river Vecht, that was of vital importance for trade to the city of Utrecht, about 40 km upstream.
The ruling bishop of Utrecht wanted to roll back Holland, of course, but at the time his military possibilities were limited. So other ways had to be found.
Luckily for Utrecht there were discontented nobleman in Holland. One of them, Gerard van Velsen, would later be immortalised in stories and ballads for what was to follow. It is certain he had a conflict of power with count Floris. Possibly his wife had been raped by the count, another good reason. While on the background there were national and international political intrigues around count Floris. Involving the bishop, but also the kings of France and England. As always in war with each other, and fearing that Holland would switch sides.
During a falcon hunting party in 1296, van Velsen and a few others captured the count, and jailed him in his own brand new castle, here in Muiden.
As Holland had been granting privileges to the small towns in the region, making itself popular, armed citizens came to the rescue of their count. The noblemen tried to get away with their hostage, but they were attacked not far from the castle. Seeing no way out, they killed the count.
A legend was born.
Using the occasion, the bishop of Utrecht took over again, and the story goes that he destroyed the place and took the stones with him to build a new castle, halfway the river towards Utrecht. It is a fact that stones and bricks were extremely valuable in those days, and were often recycled many times.
Anyway, some 100 years later, the castle was rebuilt or repaired, and that's basically the one we see today. By then this northern part of the river was firmly in the hands of Holland. A short stretch only, because even today the border with Utrecht is only 6 km to the south. But this was enough to close off the river and to ruin Utrecht. Very much to the benefit of an upcoming and very ambitious little town : Amsterdam.
When you visit the castle, don't forget the gardens. There's a vegetable and a herbal garden, set up the way they were in the early 1600s. Even in those days the castle had to be largely self supporting, so gardens were important.