I just caught my landlord trying to enter my apartment. What should I do?

I happened to take my headphones off when I heard a key going into the door over and over. I was startled because I didn’t know what that sound was, until I went to the door and looked through the peephole and found my landlord trying to use a key to open my door. (Apparently he didn’t know I was home.) I opened the door just as he was about to and he told me he was passing out a survey he needed to have me complete for the loan for the building. That was all he needed. I didn’t act upset because I was taken by surprise and didn’t have a chance to think about it, but I KNOW you can’t just enter a unit without giving notice, unless it’s an emergency, which this was not. (I should also point out this landlord is an attorney by day, so he of all people should know he shouldn’t be doing this.) I feel violated and angry. What should I do? How would YOU handle this situation?