I've always found it awkward to describe ratios as "n times fewer." Having a multiple when describing a smaller value doesn't seem right. For example, if one value is 2 and the other is 8, one might describe the first value as 4 times fewer. I've always preferred saying it one fourth as much, or describing the second value as 4 times more.
Actually, I don't like that either because of the use of "more" and "fewer." Those seem additive and subtractive rather than multiplicative. Consider "20 oz more." That means adding 20 oz to whatever the base amount is. "4 times more" implies you have your base to which you add 4 times the base, for a total of 5 times the base. I'm sure that is at least occasionally the intent, but it doesn't appear to be the usual meaning when using that phrase.
I prefer using "as much" instead, which is multiplicative rather than additive. "4 times as much" is clearer. That's another reason why "4 times fewer" seems awkward; you take 4 times your base and subtract it, leaving -3, which usually makes no sense at all.