After May 1st, if you question the legitimacy of someone identifying themselves as a census worker and they present a Census ID (how many people actually know what a legitimate Census ID looks like???) you can ask them to show you another ID to make sure the names match up on each. All workers will have a census Dept ID and since most people will not know what a legit census badge looks like it may be prudent to ask for the 2nd ID.
The 3rd check is that the census worker at their door should have a telephone number with them to provide to the homeowner to verify that they actually work for the census. The verification numbers are based on where you live so they would be too numerous to place here but for those numbers and general census information call 1-866-872-6868.
[Thanks to Kevin for the info.--former Federal Law Enforcement Officer]