The National Weather Service Tuesday issued a cold weather advisory for parts of the Delmarva Peninsula, including the lower Eastern Shore of Maryland.