Problems Persist With New 2016 Accelerate Okanagan “Tech Industry Analysis”
Accelerate Okanagan should be commended for publishing a document, the stated goal of which is to “assist in attracting new talent, companies, and potential investors to the Okanagan, as well to inform policy makers and the media.” Such reports are commonly used to promote a community or region’s economy. However, as with the earlier 2015 report, there are persistent issues, particularly with the industry definition and methodology of the study. The result is questionable data and numbers that simply do not pass a basic “sniff test.” Accepting the results of this study as published may only serve to mislead community leaders on planning, and mislead prospective entrepreneurs considering relocating here.
I taught Industry Analysis at the University of British Columbia, and my entire career has been in high-tech in Silicon Valley and globally, beginning with many years at Intel Corporation, so my assessment is exclusively from a professional perspective. A PowerPoint presentation of my work in this area is posted on this website, under the heading Professional Stuff.
The report begins by explaining that the study was completed by an unnamed third party, apparently affiliated with Small Business BC. A review of the Small Business BC website, staff, and services indicates the organization is almost exclusively organized and resourced to provide services only to individual small businesses. For example, scanning SBBC’s “Market Research” heading, it indicates that its services are focused entirely on smaller scale research for an individual small business, not a large scale analysis of an entire industry in a region. Industry analyses of such scale are better suited to a local educational institution like UBC, with all the requisite skills and resources. Though I have no inside knowledge, it seems reasonable to surmise that some degree of budgetary constraint and political influence were involved in the selection of SBBC, and a desire to emphasize local promotion over objective accuracy.
With regard to methodology and industry definition, the Report states that it follows the methodology of British Columbia’s High Tech Sector Report, the most recent of which is from 2014. A closer look at this methodology can be found on the provincial government website. A separate document is listed, “Defining the British Columbia High Technology Sector Using NAICS,” published fifteen years ago in 2001. My review of this document indicates that while it offers some useful discussion, it is seriously out of date and in need of revision. A more professional approach would have required the development of a more current methodology relevant to the Okanagan situation. The BC methodology document does provide some very cogent cautionary remarks on high-tech industry definition and methodology:
The “high technology” sector is a popular subject of discussion and analyses, partly because it is viewed as an engine of growth both in the past and for the future. However, the high-technology sector has no specific and universally accepted definition. Defining and measuring the high technology sector can be done as part of basic research at the level of individual firms. A second, more “modest” approach uses pre-existing data collected on “industries” which are defined for general statistical purposes. The challenge is to determine which of these industries warrants inclusion in the measurement of the high technology sector.
The AO Report author seems to have accepted both approaches. Page 4 of the Report explains that the author decided to also include “the previous survey undertaken by Accelerate Okanagan.” The previous AO survey was simply a Survey Monkey survey submitted by individual local businesses. The results were apparently compiled without additional professional judgment applied, or follow-up contact with companies by phone or other means and cross-referencing with the more “modest” macro data methodology mentioned in the 2001 BC document. IMHO, if my assumptions are correct, the Survey Monkey data should have been thrown out as unreliable, or regenerated with much greater scrutiny and judgment applied.
Then there is the issue of Kelowna as an employment market, as noted in the recently reported Bank of Montreal (BMO) and BC Business low national and provincial rankings of Kelowna’s employment market. These issues have also been reported in KelownaNow. Hootsuite, whose founder is from Vernon, consciously chose Vancouver to start his company. CEO Ryan Holmes openly admitted that he did not base Hootsuite in the Okanagan because he knew he would not be able to attract the necessary talent here. It is also important to note that a significant number of local business and community leaders met with the BC Labour Minister and reported that their primary concern was a lack of Temporary Foreign Workers, not economic development or the growth of the local high-tech industry.
The AO Report touches on these issues only very tangentially and indirectly in the closing pages. A more credible approach would have been to confront these local problems directly, citing the BMO report for example, and what AO and the community plan to do about it. Clearly, there are unresolved and ignored contradictions with the AO report that damage its credibility and usefulness.
Finally, this week’s media coverage of the report has died down, having duly reported all the desired sound bytes, but a Google search shows that the media coverage has so far been nearly exclusively from the local Okanagan media which does not meet the stated goal of the AO effort to broadcast the promotion beyond the Okanagan.
Read the complete AO September 2016 report here:
Click to access Economic_Impact_Study_2015_Edition.pdf
MAYO615 REPOST from January, 2015:
AO Tech Industry Report Lacks The Rigor Necessary To Give It Much Credibility
Read the AO January 2015 press release and access the full report here
The AO report’s “economic impact” conclusions are based on 2014 Survey Monkey voluntary responses, which are problematic due to an apparent lack of critical assessment. The report does not follow the kind of rigorous industry analysis performed by leading technology consultancy firms like International Data Corporation (IDC) or Gartner. The definition of an “industry,” for example the “automobile industry in Canada,” involves broad activity around all aspects of “automobiles,” but at some point, firms like Kal Tire or “Joe’s Brake Shop” might be excluded from a definition of the automobile industry. The report does not mention the rigor applied to this industry analysis, so the question is left open, “What exactly is the “tech industry” in the Okanagan?” A well-defined $1 Billion industry is the mobile advertising industry in Canada. Is that what we have in the Okanagan? By way of comparison, I reported on New Zealand’s Ice House tech incubator economic impact report, which has much greater credibility. The AO report is essentially claiming that the Okanagan technology economy is more than twice the size of New Zealand’s…That’s too big of a leap of faith for me. Read New Zealand’s Ice House Startups Achieve Impressive Results and contrast it with the AO report.
Then there is the issue of Kelowna as an employment market, as noted in the recently reported Bank of Montreal (BMO) and BC Business low national and provincial rankings of Kelowna’s employment market. These issues have also been reported in KelownaNow. Clearly, there are unresolved contradictions with the AO reports.
Read More: Kelowna’s Low Jobs Ranking
Read More: Okanagan economy likely to worsen next year
I offer a summary view of “industry analysis” here: Industry Analysis: the bigger picture
mayo615 