My 4.6l engine was doing the same thing and a slow oil leak but a new Oil Filter Adapter Gasket FEL-PRO 70801 on the side of the engine fixed it all.
That’s interesting because that’s one of the parts I got replaced when I bought the car. Had a small oil leak and they replaced the oil filter housing gasket and it solved it. But unlike Logan said, I’m not sure how they’d be connected, as mine is still doing it,while not having any oil leaks.
Did an engine oil analysis test recently though, said everything was perfect but it did show 0.2% contamination with ethylene glycol. Super tiny amount but obviously it’s supposed to be zero in there.