This is the only fight in the entire game I was unable to win.
I have always relied on White Raffards decoction to gain instant full health during a fight
and white honey to instantly remove all toxicity.
In this fight the game will not allow me to use either.
Well done to those who won.

Just a matter of figuring out his pattern.

Yes, but I found the time it took for me to reach him while his pick was stuck in the ground left me no time to inflict damage.
Possibly I should have dodged instead of rolled.

Wait you are literally unable to use the healing potion???? o.O Don't dodge, roll; his shovel has way more area then it appears to have and he can rotate while attacking.

I also found this boss to be unfair... you CAN'T fight him head to head.
The only way to beat him is to hit him from behind each time the shovel is in the ground, then once he pops the shades to kill for healing himself.... run up behind him and pop thunderbolt, etc and just wail on him.
Btw the caretaker is Relic type, if you want to use blade oils too.

I found it more effective to just kill the shadows before he reaches them. They're a one-hit kill and it prevents him from replenishing his health. You just have to keep an eye out for him, as he'll walk towards the shadow that is the closest to him. You can see easily anticipate where they'll spawn by following the foggy black trail they create.

I tried that way.
The problem is he just constantly spams that medium sized aoe knowckback spell when there's no shadows.

If you leave the shadows, he'll very slowly walk towards them, while completely ignoring your attacks.

It's all about timing. Depending on how fast you get to him after he gets stuck after his charge attack, get in 2-3 hits and immediately roll out of his aoe radius.

They meant it to be a long epic fight.. You can really only get a 2-3 hits at a time in on him, but eventually you can wear him down if you don't let him damage you, and you kill the ghost before he does. It helps to be able to heal a lot so Ekhinda decoction and Whirl or Ekhinda decoction. Archgriffin decoction and thunderbolt all will help you get throw.

No one has referred to my original stated problem of not being able to regain my health with white raffards decoction
nor being able to remove my toxicity level with white honey like I can do in any other combat.

I found I could fight him head to head. You can step aside during his charge attack and hit him 3-4 times in the back, as well as or instead of relying on the 'Imlerith' pause. He does also break off to collect, and you can safely hit him from behind while he is harvesting.

Getting hit is a bad idea, but not that hard to avoid if you just stay out of reach until he over commits in one way or another. Fleet footed and Quen help out immensely in this fight. You seldom get hit, while dodging and even if you do it does no damage = no regen.

I found I could even dodge in, take a single fast strike and dodge out without getting hit as well most of the time.