Should I break up with him? ?

That is a really complicated situation. Don't break up with him, sit him down and just lay it out for him. This is how it is now, this is what I want because it's just not working for me. Tell him right there if this doesn't change in the next month get your *** out of my house because we will be done. Be completely honest with him and if he has a problem with it than he isn't worth it, if that's the case you should dump him.