I do the same for the same reason.
I can knock out a basic detail in 4-5 hours-full interior detail, one polishing step/clay horizontal surfaces and wax. Been doing it so long I guess I just have the process down so well it is automatic. Heavy correction though, it takes as long as it takes.
Funny thing is, if I check the time before I start a detail, I know what time it is (usually within 5 minutes) by what point I am at in the detail. Once you get down the process that works best for you, stick with it and your details will go faster without compromising quality.