My suspicion is that you have had some history being cheated on? Jealousy or ‘paranoia of cheating’ are foreign concepts to me, in general, unless there is actual proof of suspicion (i.e. another girl and He are cuddle buddies or something). I have always kind of felt – if some chick is dumb enough to cheat on me, then well – she is dumb. Fuck off. On to better pastures.
First, why are you not going to the party with him? Second, is there is shit you are saying like: “He has never done anything to me to make me feel paranoid” but that is more to convince yourself and not me? You don’t normally feel paranoid if you are both open and vulnerable with your feelings. Is he open and vulnerable with you? Is your relationship equally reciprocated? I am guessing, that you have some shit to communicate about that isn’t really “will he cheat” but more “does he care as much as you do”.
Never settle. That is my biggest piece of advice – communicate – and expect the same that you give in return. If you don’t get it, end it and move on.