Back in figure drawing class in college, I used my eraser a lot!! At break, us students would go outside and rub our erasers on the brick wall to get all the black off and renew them. You could do that, or you could use an emery board like they did HERE.
Do It And How 