"It had everything to do with him," James confirms. He cannot and does not want to condemn the Master entirely, even if he condemns his actions. But he had his reasons, and he's part of James. That doesn't mean he approves of anything that hurts other people. But the Master never truly considered Earthlings people.

"Though not with her, really. Poor girl, she was just as much as a victim as the rest of us. Probably even more so." Fondness for Lucy is probably another thing James retains from the Master - even though they didn't interact all that much.