Janet Malcolm policed the literary ruses of others, but wasn’t afraid to deploy them herself.