Independent Inflation Tracking Numbers
Updated February 19, 2016
For some reason people don’t seem to trust the government. I can’t understand why. Surely the government only has our best interests at heart and wants to take care of us like good parents, and they are just protecting us from ourselves. And of course all politicians are honest, selfless, hard-working civil servants. Right?
Well, Okay maybe they don’t always have our best interests at heart. And maybe it would benefit the budget if they didn’t have to pay so much for cost of living increases but surely they aren’t fudging the Consumer Price Index are they?
I frequently get emails, and occasionally phone calls, asking just that question. Often the conversation will go something like this: I know that a few months ago when I went to the grocery store my favorite Arnold Oatnut bread cost $2.50 and only a few months later it is costing me $2.89 so lets see that’s a 15.6% increase in six months so annual inflation must be around 31% right? And the government is telling us that inflation is around 2% so they must be lying!
I recently got a call from someone who was convinced that the CPI didn’t reflect reality because the government has started leaving out food and energy. This is a common misconception because you can look at the “Core CPI” which doesn’t include food and energy. The core CPI though is primarily for statisticians who are trying to track the effects of increases in the money supply and food and energy are often subject to other outside forces including everything from the weather to unrest in the Middle East.
One reason for the possible confusion is that the Federal Reserve uses a method of calculation called Median CPI which is similar to core CPI for determining its monetary policy. To calculate the Median CPI, the Federal Reserve Bank of Cleveland looks at the prices of the goods and services published by the BLS. But instead of calculating a weighted average of all of the prices, as the BLS does, the Cleveland Fed looks at the median price change—or the price change that’s right in the middle of the long list of all of the price changes.
So if you are a statistician you can look at the core CPI, but otherwise it is pretty much worthless, and for specific purposes you can also look at just food or just energy or almost any other combination you like. But if you want to track the entire 10,000 item list of goods that the “average” consumer might buy you can look at the “Consumer Price Index for all Urban Consumers” (CPI-U) or just “CPI” for short.
Other Fudging Possibilities
Even though the CPI doesn’t exclude food and energy like many people think, it is still possible that the government is fudging the numbers. John Williams at Shadow Stats contends that the government has changed the way it calculates the CPI over the years and thus it is chronically under reporting the inflation rate so he tacks on a flat 4% to the government numbers to make up for their “under reporting”. Some how a flat addition doesn’t sound like the right approach to me either.
I have always wished I could track a bunch of items and have my own price index. Then I could compare it to the CPI. Then I would know exactly how the CPI stacks up and I’d also know how my purchase prices have changed over time. Apparently I’m not the only one… recently two economists, Alberto Cavallo and Roberto Rigobon at M.I.T. have developed a brilliant way to do just that. As a matter of fact, their method is so ingenious that it actually tracks inflation on a daily basis instead of just monthly. Rather than doing it like the government does and hiring thousands of workers to call businesses and find out the current price of everything from apples to Zip-Lock bags, they have developed a system that tracks online prices and calculates the inflation rate.
They freely admit that it won’t be exactly the same as the CPI since it only tracks online prices but it does track supermarkets, electronics, apparel, real estate and many other things. Unfortunately it doesn’t track services because they are generally unavailable online. At this point they are actually tracking over 5 million individual items compared to the CPI’s 10,000 so for the categories it tracks, it actually provides a greater level of accuracy in addition to providing data daily instead of just monthly.
Cavallo and Rigobon are working on making it a world-wide index. They are currently tracking prices in over 70 different countries. Of course, many of those items don’t apply to the U.S. inflation rate because they are priced in Rubles or Lira or something other than Dollars. But you can limit your search to just the United States. So that is exactly what I did. Since they track it on a daily basis I compared their data on the last day of the month with the CPI for that month (which comes out two weeks later from the government). Cavallo and Rigobon call their index the “Billion Price Project” or “BPP”. (Note: Since this article was originally written the BPP went “for Profit” so data is only available via subscription through State Street Research.)
State Street Research now maintains and markets the data to companies needing more reliable data than that provided by the government. If companies are willing to pay for data they could get for free what does that say about the accuracy of the BLS data?
In the following chart we can see that their BPP index (called the Daily Online Price Index which is their version of the Consumer Price Index) has been consistently higher than the CPI during inflationary times. This lends credence to the idea that the government is under-reporting inflation. As we can see from the chart the difference is cumulative and continues to grow as time goes on. If it were just a statistical anomaly the BPP index should cross below the CPI 50% of the time.
Although it is obvious that the BLS is under-reporting the index, unfortunately it is difficult to tell by looking at the index itself how significant the under-reporting is. But if we look at the Annual Inflation Rate we can see some interesting things. Unfortunately, due to their publishing delay we are unable to see the most current numbers.
Looking at the chart above you will see how the two compare. Note that the BPP numbers (called “Online Price Index”) track the government numbers quite well sometimes and very poorly for most of 2010 and 2013-2014, where the BLS was significantly under-reporting inflation. Interestingly, in 2014 it appeared that the BLS was finally returning toward reality (although still understating inflation). I find it interesting that as deflation took hold the lines crossed and now the BLS is actually under-reporting deflation. This makes one doubt an inherent bias to the downside in the BLS calculations and presents the possibility of a much more intentional fudging of the numbers.
I have received some flack for reporting on the BPP with some people complaining that it doesn’t include services. And that is a very good point. Obviously, it is difficult to buy services online. At least services like healthcare and plumbers. Although more and more services are being offered on sites like elance, Guru.com and fiverr. But the point that healthcare and education aren’t included could be an issue, after all the BLS readily admits that both sectors are growing faster than the average. But if these two highly inflated sectors are being under-reported in the BPP the inflation gap is actually more than 1% in 2010 and 2013-2014.
In addition to inflation numbers, there is evidence that the government is fudging other numbers as well. See Employment vs. UnEmployment to see how the government’s own numbers don’t agree. Also see Is the Government Fudging Unemployment Numbers? to see a comparison of the Government’s numbers to those collected by the Gallup poll people. See also More “Funny” Data from the Government where Leonard Brecken of Oilprice.com calls into question numbers generated by both the International Energy Agency (IEA) and the Energy Information Administration (EIA) regarding oil production and demand.
Personal Perspective on Actual Inflation:
In early 2012, I happened to be in the grocery store and was thinking about prices back when I was in college in the late 1970’s. My roommates and I used to do the grocery shopping together and buy the little 8 oz. cans of tomato sauce (not the standard 16 oz size). We always bought the cheapest stuff we could find. Store brand or “White label” and for some strange reason I still remember the price… 25¢ a can. So when I was in the store I happened to see the little cans and guess what… the store brand was 35¢ and the white label was 25¢ a can. This is 35 years later and the inflation on a can of tomato sauce is between 40% (@ 35¢) and zero % (@ 25¢).
Even if the total inflation is 40% the Compound Annual Growth Rate (CAGR) is only 0.97% (the same as if the annual inflation rate was 0.97%). Of course tomato sauce (along with milk and eggs) are often “Loss Leaders” for grocery stores and their prices probably don’t reflect the true level of inflation in the economy.
See: Investopedia’s Compound Annual Growth Rate Calculator
So maybe tomato sauce isn’t a good example. So from the grocery store I happened to go to Sam’s and there I was amazed to find a microwave for $69. I think my parents paid about $700 for theirs back in 1978 (admittedly it was much bigger). But I’m sure I couldn’t buy any size microwave for $69 in 1978.
In 1981 the year after I graduated, my college was the first college in the country to require all incoming freshman to have a personal computer… an “XT” if I remember correctly, monstrously heavy desktop, generic brand, 256k RAM memory, school discount pricing, $3000.
Today you can get a laptop with 4 gigabytes of ram and a processor that is many multiples faster than an XT for about $500 without any special discounts and recently Walmart had desktops for $300. (add another $100 for a thin LED monitor 2 – 3 times larger than anything available in 1981.)
So a lot depends on which products you are looking at. Where’s your focus? Is the glass half empty or half full? Are we getting killed by inflation or are things incredibly cheap? If you don’t trust the government, I don’t blame you. I don’t trust them either. But the folks at MIT have less reason to fudge the numbers, they aren’t paying out billions based on their calculations. In fact, they are using the data for research purposes (and selling the data to corporations and market traders through “State Street” ) so they have incentives for getting the most accurate data possible. They also have technology on their side as they use the Internet to automatically collect prices. And since corporations are willing to pay high prices for the same data they can get for free from the BLS my guess is that the corporations don’t trust the government data either. So what do you think? Let us know below.
See Also:
- What is Core Inflation and Why Doesn’t It Include Food and Energy?
- Is the Federal Reserve Right About Inflation?
- Impact of Inflation on Bonds Part 1 Part 2
http://www.shadowstats.com/article/no-438-public-comment-on-inflation-measurement.pdf
Here is a great article with five bullet points on why current inflation/CPI numbers are phony.
My vet recently posted a sign that because of increases in inflation he now must have payment at time of service. That’s a 70 year family owned veterinary office and that sign hardly indicates tame inflation.
Nice try but…
There’s a couple pretty big factors that I haven’t seen taken into consideration by edu, gov or here.
One is what I call inflation by deflation and it’s something that many I know have noticed but apparently it’s been going right past most since it’s seriously under discussed.
Here’s what inflation by deflation is… Start to really look at the weight and volume of what you’re buying and if you can (or can remember) compare it to the same product a few years ago. Look at the size of the candy bar, the bag of dog/cat food, frozen veggies, etc. Basically inflation by deflation seems to be a marketing strategy that many manufacturers have come up with to make us feel like the prices we pay are not as bad as they actually are. Unless the gov and the study above use actual weights and compare a pound to a pound, ounce to an ounce and a quart to a quart, they’re probably inaccurate.
Now here’s the second factor… the CPI used for Social Security doesn’t reflect the reality of what seniors and the disabled actually spend money on so the amount needed by them to cover THEIR areas of inflation is often much higher then the CPI reflects.
Please go do some reality based research on what most Americans reality actually IS. Unfortunately for those who have no choice but to live by the CPI too many that should know, apparently haven’t a true clue (or are deliberately misleading).
I agree that getting smaller and smaller packages is just an effort to fool the consumer. But the BLS does take this into consideration when they calculate inflation. And the BLS does calculate a seperate “Senior” inflation rate based more on what they might be purchasing. The CPI-U is designed to estimate the inflation rate experienced by the “Average” American but obviously very few people are “average”. Everyone is unique in some way or other. So all you can do is create your own inflation rate by tracking things you actually buy regularly. You might also be interested in a comparison of government Unemployment numbers compared to those gathered by Gallup Poles. Is the Government Fudging Unemployment Numbers? http://unemploymentdata.com/2012/04/07/is-the-government-fudging-unemployment-numbers/
If the Cavalo and Rigoban index excludes services then their work can be tossed out due to IRRELEVANCE. It’s the services that are causing the price increases in healthcare, dental care, vet care, and higher education.
Mr. McMahon: How can you endorse such an index that does not track services?
Mr. John Williams is right on the money.
Mr. McMahon – all due respect – but price increases are price increases are price increases. They deserve no qualification– insurance health care does not change one bit the fact that health care prices are presented to the market and paid by the market. All costs are paid by individuals ultimately.
A price presented to the market and borne and paid by the market is the price. The fact that the dollars exist to pay that price are the only thing that matters- THE ONLY THING THAT MATTERS. That defines inflation and on a monetary policy level relating directly to Federal Reserve policy and not some phony CPI number.
RE: Housing – In 1963 my family bought a house for $13,000. That house TODAY, NOT 2006, sells for roughly $275,000 in spite of the housing boom/bust. So the last 5 years is an anomaly and housing is just reverting back to the upward trajectory it has been on since the 60s. That’s the planet I’m on :).
It is the inflation qualification mindset (with Chain CPI being the latest nonsense qualifier) that has allowed excess liquidity policies and the proof is the unprecedented investment bubbles and our best inflation gauge gold which is nominally higher than the cold war (with massive nukes) and will exceed that amount soon. Hedonics (your spellchecker does not even know the word hedonics)is a SYMPTOM of an inflationary environment and not a discount to it. All of this stuff is an invention of Alan Greenspan playing the economy like his own personal Milton-Bradley board game, ignoring and manipulating free market forces and feedbacks, and hence the Great Greenspan Depression we are now in.
Don’t forget veterinary care and dental care as well. Housing, education, vet care, dental care are all core costs for all people.
Money supply is not the only aspect of inflationary policy.
If you know the history of shadowstats and Mr. Williams, it/he came about because business quantitative models went haywire with the qualified inflation, GDP, productivity, you name it, numbers. His models explain the qualification/corruption of the government numbers.
The MIT guys? I need to know their funding sources before I will even research the work they are doing.
Been studying this daily for 3+ years and will continue. I enjoy the discussion really but I think I’m right and I am close to saying that I KNOW I’m right.
.
Inflation in housing? What planet do you live on? Housing prices peaked in June of 2006 and have been falling steadily ever since. See the following Zillow Article:
http://tinyurl.com/3jkot6m
The chart shows average home prices in June 2006 at $240,683 currently it is $172,617 a net DEFLATION of 28.28% in spite of QE1 and QE2. That is roughly -5.6% a year. The Zillow article states “The monthly increase in May of 0.01% was the first positive monthly gain at the national level since the onset of the housing recession in June 2006 (58 consecutive months of monthly declines in home values).”
I agree that health care and education have experienced massive outsized inflation but neither is a monetary phenomonon. Both are due to government interference in the form of increased regulation and government grants. The increase in education is due to the availability of student loans to everyone no matter their ability to pay. Recently it has been reported that homeless people have been recruited by the online giant University of Phoenix. They don’t care if the loans will be repaid it is just Gov’t money. Same with the giant health care/ insurance industry scam. Without the prolifferation of insurance healthcare would be subject to market forces and prices would be held down as doctors would have to compete based on price. Insurance has eliminated the market forces. And interestingly also the same with the housing industry prior to 2006 thanks to Fannie and Freddie. All government created bubbles due to regulation and subsidy not due to strictly monetary forces.
Food and clothing are the two primary recipients of true inflation these days and yes they have increased in price drastically. Other than the wierd anomally of tomato sauce that I mentioned previously.
Yes computers are subject to Moore’s law which is the result of productivity overcoming the effects of inflation through technology and increased productivity. But you ignored my comment about microwaves which last time I looked were classified as a home appliance not a computer.
They probably benefit from the improvements in technology as well but if you exclude them then what? Exclude everything that goes down in price and only include things that go up? My point is not that the government is telling the truth they seem to be understating the inflation rate by about 1% based on the independant studies (not 4% like Shadowstats says). And cheap production in China and technological advances were helping to mask the increases in money supply but the money multiplier has been negative for a couple of years now and so no matter how much debt the government creates or money it prints it is like “pushing on a string” and the money supply is not expanding at the rate it would if people were in an expansive mindset and defaulting debt wasn’t contracting the money supply.
Thanks for the reply. I find it interesting that your inflation examples are tomato sauce and computers and not health care, dental care, higher education, and housing. Those sectors having experienced tremendous inflation.
The computer phenomenon is a function of Moore’s law and is unique to computer manufacturing. Further, there’s not much relevance to the price of computers if I can’t buy and use an AT today. It is a useless piece of machinery. I hardly think you can use uselessness upon which to base a current price. It’s a little like relating the price of a modern automobile with a horse and buggy.
“Moore’s law describes a long-term trend in the history of computing hardware. The number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years. This trend has continued for more than half a century and is expected to continue until at least 2015 or 2020.” — Wikipedia
We have malicious inflation today. The Federal Reserve, which produces nothing other than buying stocks, driving up the stock market, and then selling those stocks– has purchased 90% of U.S. debt over the past year to keep interest rates and inflation down artificially. We have deserted the free market and there will be a price to pay for it.
Again, the price of gold is extremely revealing re: the value of currencies which are being printed at an enormous rate and inflation.
Enter Ron Paul and Ben Bernanke into youtube and you can be schooled along with Bernanke by Dr. Paul.
I’m sure it has to do with the level of processing. Minimally processed necessities like canned tomatoes are marked up much less than discretionary items like Doritos. Farmers would say it is discrimination against farm products. Inflation certainly isn’t uniform across all products and Doritos is not in the same category as tomatoes. The cheapest I remember Bananas is $.25 cents a pound and they stayed there for many many years. Now they seem to be around $.49 to $.59
BTW. I don’t remember Doritos at $.49 I do remember them at $.99 but I rarely buy junk food. Also bag sizes are constantly changing. What about Pepsi products? Seems they got drastically cheaper at one point several years ago and now have started inching up again. Since it only costs them a few cents to make they can mark it up anything they want and still make a profit. So it all depends on demand. Same with Doritos.
According to Xomba “Doritos Prices Rising Faster than Gasoline” http://www.xomba.com/doritos_prices_rising_faster_than_gasoline due to the use of corn in Ethanol. So now we have gasoline competing with Doritos for the available corn supply.
When I was in college in 1971, I bought a bag of Doritos (Taco) for .49. Today that bag is not .49. It is $3.99. I think (correction, I know) you are mistaken about the tomato sauce unless it is made in China.
MIT not fudge the numbers? It happens all the time to get the politicians to give you the right amount of research money every year. Get out of fantasy land Mr. McMahon.
Listen,
17 years of business expansion (1983-2000) and no problematic inflation? Not likely.
An unemployment rate approaching 3% in the 90s and no problematic inflation? Not likely.
The dollar crashes and no problematic inflation associated with foreign goods? Not likely.
Unprecedented stock, housing, and commodity bubbles (commodity bubbles in my mind DEFINE problematic inflation) over the last 15 years and no problematic inflation? Not likely.
Price bubbles in housing, health care, higher education at the same time and no problematic inflation. Not likely. (BTW, COBRA has been rendered useless by health care inflation). (Housing: no pressure on the credit markets during the boom? 30 year peaked at 6.9% or so. I would love that mortgage rate in 1986)
Niall Ferguson says the inflation calculation has been changed 28 (28!) times since the 70s. Wouldn’t the Weimar Republic like to calculate inflation the way we do now? If we calculated inflation in 1980 the way we calculate it now, Jimmy Carter is a two term President.
If it walks like a duck and quacks like a duck, I do not need to check its DNA (ref: flank steak vs. filet mignon, 10,000 items (what a ruse – 10,000 items cannot be wrong says the BLS), et al), it’s a DUCK!
Mr McMahon’s facetious start to the answer regarding trust in the government numbers shows promise for him if he realizes the accuracy of it.
I have been studying this issue for many years. My conclusions in a nutshell:
The CPI is a hoax courtesy primarily Alan Greenspan who destroyed the U.S. banking system (find yourselves a Federal Reserve bank examiner) and the world banking system. Unfortunately, regardless of the CPI’s true technical meaning, it is sold to the public as an indicator of malicious inflation. We have malicious inflation in every sector whose production cannot be shipped to China or cannot take advantage of the efficiencies of the internet. Those sectors include health care, dental care, veterinarian care, and higher education among others.
We are told that health care prices are spiraling out of control to justify Obamacare yet the core inflation index is 1%? That does not square.
The BLS uses hedonics, substitution, and weighting to “modify” inflation. None of these are valid. In fact, my own personal theory is that hedonics is a SYMPTOM of an inflationary environment and not a discounter to the inflation rate.
The reason for the “modifiers” of inflation is primarily to artificially reduce the payout on Treasuries especially Inflation-Protected Treasuries (TIPS) which I regard as the biggest sucker deal in investing. The “modified” CPI is also used to reduce Social Security payouts.
John Williams (the economist and not the San Fran Federal Reserve John Williams) is absolutely correct.
The price of gold is not a bubble. Bubbles don’t last 10 years– 10 years is a trend. The gold price is an indicator of several things but most of all it is the value of fiat currencies especially the dollar.
In a larger context, you cannot provide massive amounts of liquidity to the economy without inflation. The Federal Reserve has pursued excess liquidity over the last several decades ostensibly to fulfill the employment aspect of its dual mandate. We are currently at 9% unemployment (yet another “modified” government statistic) in that mandate. Good work boys!
Boldhawk:
The BLS disagrees with you on that one. The key phrase in their answer is “Substitution is only assumed to occur within basic CPI index categories, such as among types of ground beef in Chicago. Hamburger and steak are in different CPI item categories, so no substitution between them is built into the CPI-U or CPI-W.”
Full text answer on their Common misconceptions page they said:
Question: When the cost of food rises, does the CPI assume that consumers switch to less desired foods, such as substituting hamburger for steak?
BLS Answer: No. In January 1999, the BLS began using a geometric mean formula in the CPI that reflects the fact that consumers shift their purchases toward products that have fallen in relative price. Some critics charge that by reflecting consumer substitution the BLS is subtracting from the CPI a certain amount of inflation that consumers can “live with” by reducing their standard of living. This is incorrect: the CPI’s objective is to calculate the change in the amount consumers need to spend to maintain a constant level of satisfaction.
Specifically, in constructing the “headline” CPI-U and CPI-W, the BLS is not assuming that consumers substitute hamburgers for steak. Substitution is only assumed to occur within basic CPI index categories, such as among types of ground beef in Chicago. Hamburger and steak are in different CPI item categories, so no substitution between them is built into the CPI-U or CPI-W.
Furthermore, the CPI doesn’t implicitly assume that consumers always substitute toward the less desirable good. Within the beef steaks item category, for example, the assumption is that consumers on average would move up from flank steak to filet mignon if the price of flank steak rose by a greater amount (or fell by less) than filet mignon prices. If both types of beef steak rose in price by the same amount, the geometric mean would assume no substitution.
In using the geometric mean the BLS is following a recognized best practice for statistical agencies. The formula is widely used by statistical agencies around the world and is recommended by, for example, the International Monetary Fund and the Statistical Office of the European Communities.
Source:
http://www.bls.gov/cpi/cpiqa.htm#Question_3
In addition to Tim McMahon’s information, one will also find in the Bureau’s web site an explanation of how the CPI is calculated.
It is interesting to note that during the Clinton Administration the famous market basket was changed to contain variable yearly, to “flatten” the inflation, by including substitutes for items that had gone up in price, assuming that if someone didn’t have the money to buy the more expensive item, he would replace it with a less expensive item. In other words, if a cut of beef went up in price, then, they assume that people who can no longer afford it, will buy kidneys, or liver,
Another aspect is that if one includes a big ticket item, such as homes, and homes have gone down in price significantly, then that item will flatten the CPI to show no inflation. In fact, because of it, if it was permissible to go negative, the drop in home prices have been so steep that they would produce a negative CPI… They haven’t gone down that road yet, but if history is to be an indicator, they will, and Soc Sec, and other entitlements that are increased by COLA, will also be decreased.
The obvious thing is that not all people receiving Soc Sec will buy a home this year or next. So the application of the CPI is not in accordance with its purpose, i.e.: even out the commercial playing field, so people do not lose buying power over time.
What is tracked for the inflation number issued monthly?…what is not?
Jim,
The Bureau of Labor Statistics includes over 10,000 different items in its CPI index.
Yes, including food and energy.
You can see the complete list of all the categories (but not all the individual items) on their site at http://www.bls.gov/news.release/cpi.t01.htm
Isn’t it curious that they don’t publish the entire list? Where can i view the entire list and their weighting formula? Thanks Tim
The BLS publishes a fairly extensive list of the individual components of the CPI-U compared to the CPI-W and their respective weightings here: https://www.bls.gov/cpi/tables/relative-importance/2016.pdf