From Grammerly:

It is also customary to use less with regard to time, even though we can count time in seconds, minutes, hours, and so on.

Example:

Ethan has been at his job for less than five years.

I wish I could spend less time on household chores.

Yet, depending on how general or specific your reference to time is, it may require the use of fewer.

Example:

I wish I could spend fewer hours on household chores and more on watching television.

---

Both less and fewer are perfectly acceptable in this context.