When the simultaneous use of the St. Servaas as both a collegiate church and a parish-church started to cause problems, the chapter decided to have a new church built right next to the old church. The new church was first used as a baptistry only, but later it became the parish-church. This church must have been finished at the end of the 12th century, but in the current building few traces are left from this period.

The nave of the St. Jan dates from the early 14th century and is clearly an example of the transition from the Romanesque style towards Gothic. While the Gothic way of architecture allowed the construction of very high churches with very large windows, this one is of a very modest height and has quite small windows, which is especially well illustrated by the clerestory. Also unusual is the nave's smooth appearance. Not only are the statues and pinacles often associated with Gothic completely absent, there's also not a single buttress in sight, let alone flying buttresses. All buttresses are positioned inside the building. There's no transept either, probably because there was no space for it anyway, with the St. Servaas being so close.

The choir and the tower are both from the 15th century, and are in a much more classic Gothic style. The tower at first sight appears to be a copy of the one of the cathedral in Utrecht, although it is not nearly as high. But apart from the general shape which was definitely copied from that of the tower in Utrecht, there are no similarities. The ornamentation is completely different. The tower was finished in 1475, and was probably designed by Jacob Herts. The choir was finished before that. It's the same width as the central aisle and has 12 sides and very narrow but high windows. At the choir it's clear to see that the church was built on a slope. This means that anyone who enters the church will have to descend a few steps.

In 1632 the St. Jan was one of two parish-churches that were handed to the protestants (the other was the St. Matthias), who still use it, although it was returned to the catholics twice, both times when the city was under French occupation, and both times for a few years only.

The church has been painted red to protect its marl stone. This is a traditional method to protect this vulnerable material, and was done most recently in 1983.