It’s become a predictable pattern. Statistics Canada releases a crop estimate. Reporters call traders and analysts for reaction, who inevitably say it’s either too high or too low.
That reflects a general dissatisfaction with StatsCan’s time-consuming system of contacting 10,000 farmers to ask how their crop is doing.
Replacing a survey-based crop report with technology-sourced data might be a step in the right direction, but a disgruntled grain trade insists more still needs to be done if the agency and its crop estimates is to regain credibility.
“My only thought is, any improvement in their forecasting would be appreciated,” says Errol Anderson, president of ProMarket Communications in Calgary. “If they’re trying different methods, that’s excellent, because the way it is, improvements are needed. We do need better information.”
In 2015, Statistics Canada produced its first modelled yield estimates as a supplemental release in advance of the September farm survey, and model-based principal field crop estimates replaced the survey in September last year.
Tony Tryhuk, branch manager of RBC Dominion Securities’ commodity futures trading division, says this is a step in the right direction.
“Eliminating bias from the equation must lend itself to greater accuracy,” says Tryhuk. “While acreage estimates and seeding intentions must continue to be survey-based, production reports will benefit greatly from more scientific inputs.”
Neil Townsend, senior market analyst at FarmLink Marketing Solutions, says farmers are tired of participating in the surveys, so anything that lowers that burden is an improvement.
“I do think the model-based report eases the burden on farmers at a busy time,” agrees Cargill market analyst David Reimann.
Not a single solution
No one, however, views the technology as any kind of panacea.
Townsend says many who made early use of satellite data have backed off their initial claims that such technology alone could provide accurate measurements, and have since used it to augment their findings on the ground.
Besides, he says, if satellite data had mixed success in the U.S., where acreage is dominated by soybeans and corn, what are the chances it will be more successful in Western Canada, where crop diversity is much greater.
“The satellite way of doing it doesn’t have enough of a track record,” adds Greg Kostal, president of Kostal Ag Consulting. “Suspect imagery and predictability is radically different between crop types.”
Reimann says data suggests the model-based information is accurate relative to typical farm survey results for that time of year, but adds those figures are usually not the best indicators of the final results anyway.
“I believe there is a place for this type of data use,” Reimann says. “In the future, maybe we will combine satellite data with farm survey results.”
How it works
About five years ago, Statistics Canada underwent a complete program review.
“We went through all the surveys and asked ourselves, ‘Are there other ways we can produce the same estimates without having to do it the traditional way, like contacting people?’ That’s when we started to work on the pilot project in trying to estimate the yields with satellite imagery,” says Frederic Bedard, Statistics Canada’s remote sensing and geomatics senior analyst.
The yield estimates are based on a model that incorporates satellite and climate data, as well as historical data from Statistics Canada’s field crop reporting series. The production estimates are calculated by multiplying the model-based yields times the reported harvested area from the July farm survey.
“We use satellite data from May to August, and we have a longtime series of satellite images that are comparable and that go back to 1987, so that’s 30 years of data,” says Bedard.
Some crops will mature earlier or later in the year depending on the crop type and province, so Statistics Canada extracts Normalized Difference Vegetation Index (NDVI) data for a four- or five-month period, and the modelling software determines which period correlates best with the yield.
NDVI is a picture of the ground. It indicates the amount, density and health of the vegetation.
Generally, more vegetation equals higher yields. That rule, however, didn’t work as well in the case of corn in particular, so climate data was added to the equation.
For the same May to August period, Statistics Canada incorporates climate station data from Environment Canada and the provinces. This data includes variables such as the number of heat units, total precipitation and stress index.
Statistics Canada’s historical crop report survey data is added to the mix.
“In total, there are 60 or 70 different variables that are available for the modelling,” says Bedard. “And then the modelling software selects up to five of the variables that correlate the best with the yield.”
There’s one model built for each agriculture region — Alberta, Saskatchewan, Manitoba, Ontario and Quebec — and then the data is aggregated at the province level for publication.
The model methodology allows for accuracy equal to the traditional interviews, Bedard says. The difference is eliminating 10,000 phone surveys, which reduces the burden on farmers and saves about $150,000 in interview costs, while also producing results about two weeks earlier.
The September report was the ideal one to replace with modelling as crops mature late July/early August in most of the country, Bedard says. Also, it gives Statistics Canada the July 31 estimates to compare to.
“The yields produced by the model are produced independently. They don’t need to match the July estimates perfectly, but we do look at them,” says Bedard. “If we do find a large deviation, we investigate to ensure it’s not an error. But it helps to have an initial estimate from another source.”
For now, the September phone survey is the only one that’s being completely eliminated, and modelling will be used again for September 2017. Replacing any other reports would require more research, which isn’t currently planned.
“I’m sure we’re going to try to see if other surveys can be eliminated, but for now at least, there’s no plans for 2017. But there’s always a bit of investigation that is done to try to fulfill this objective.”
For several years, Statistics Canada has taken a hit from the grain trade for the accuracy of its crop reports .
“I just find it frustrating when the market says ‘irrelevant’ 10 minutes after it’s released. That means something’s wrong,” says Anderson.
“You can look at the canola estimates and see how far off they’ve been several times, and then they do a big adjustment several years later,” adds Townsend. “I would say in general in Canada, we need to increase the relevancy of our statistical estimates.”
Kostal says although Statistics Canada’s figures are used as a tool in price discovery, they’re nevertheless subject to the trade’s perception of their accuracy. Like a weather forecaster’s or market analyst’s opinion, they’re right just enough to be respected, but also just wrong enough to not be exclusively relied upon.
Statistics Canada has become the scapegoat for inaccurate crop estimates thanks to “less than forthright inputs,” Tryhuk says.
“Data output being only as good as the input source, we have witnessed too many significant corrections in future production and acreage reports to have faith in the initial releases,” he adds. “Even the 2016 satellite data showed one-million-tonne higher production than the survey-based numbers collected a few weeks earlier.”
In its production estimates as of July 31, Statistics Canada pegged 2016 canola output at 17.0 million tonnes. Despite deteriorating Prairie growing conditions, that number climbed to 18.3 million tonnes as of August 31.
Shrinking sample size
Townsend believes surveys have also become less effective with farm concentration.
“The risk is if the sample size goes down, the accuracy goes down too.”
Nevertheless, Townsend doesn’t think Statistics Canada could boost its accuracy by eliminating phone surveys.
People on the ground instead of phone banks, however, could glean additional knowledge, Townsend and Anderson say.
“StatsCan, it’s more a telephone survey. That’s weakness in itself,” says Anderson, who notes USDA has far more resources, including crop scouts. “And USDA also has the backup of the ProFarmer tour… they obviously spend a lot more money.”
The ProFarmer tour is a four-day event with crop scouts collecting samples from across the U.S. Midwest and meeting to prepare a crop estimate.
By contrast to Statistics Canada, USDA has established itself as the industry’s benchmark, experts say.
“Once USDA puts out the final number, nobody really disputes that. Maybe we were there 10, 15 years ago at StatsCan, but I’m not sure we’re there right now,” Townsend says.