On windows xp on bootcamp, the delete key actually does what the backspace key should do.

To get it to act like it should, just hit hold down [fn] then delete to your heart’s desire.