Use household alcohol and a cotton ball to gently wipe the surface of a vinyl LP sleeve. This will remove dust, dirt, tape residue and some pen marks. Be careful with all sleeves as the ink will bleed on some. Try a small test spot in the corner first to test.
As for the vinyl record itself; use Windex and a clean soft cloth wiping with the grooves. Be careful not to spray it on the record label. In extreme cases you may try setting them outside somewhere dry on a sunny day (under 85 F) for an hour or two maximum. Be careful you don’t leave them outside too long or they may melt!