So - you have an uneasy feeling about your next door neighbor. He lives alone, rarely leaves his house except to go to the mailbox. And when he does come outside you get the creeps. Your children also get the creeps. This is not a good situation. You could go to the local police station to inquire about his background - but unless he is a registered sex offender, they may or may not tell you anything. Plus - the only information that they would have would be related to arrests and violations right in your town.
You could also go to the local county courthouse to look up records on this person as these records would be available to the general public, however the challenge is that criminal records are spread across every county court house in the states. You probably wouldn’t be able to do that very effectively.
To stop the insanity (and your suspicions), why don’t you run an online criminal records lookup on the guy. He’ll never know that you did it - you can use one of those online services that charge a small fee for the report. Information that you can dig up on the guy could include arrests, police reports, driving offenses, etc.
You would also be able get an address history to see if he has moved a lot (which could indicate that he is not so nice of a guy). Once you know his history once and for all, you can either stay further away from him (if he is in fact a bad guy) or decide that he is just a quiet guy who keeps to himself.