Seachem Prime will give false readings of ammonia, its actually ammonium that you are reading and its nothing to worry about as the bacteria will consume it anyways. As far as nitrifying bacteria and PH, its only in the very low range that it becomes less efficient. Here is your plan, as your water is exactly like mine, don't fret. Keep doing your water changes as normal and add a teaspoon or 2 of baking soda. This will help the KH a little and you won't have to worry about the PH dropping.
Have you aged your water overnight in a container using an airstone to see what the PH does?
Eddie