Downhill or uphill

It doesn’t make sense when people say ‘things go downhill’ because they mean things get worse. Shouldn’t that be uphill though, as in an uphill struggle?
Please observe the above picture for easy reference.
Now, discuss!
(kindly note, the author of this masterpiece might be slightly affected due to a flu fever at the time of posting this)





