The analysis of firm web data for predicting company innovativeness: a comparison across different types of innovation

Sander Sõna, Jaan Masso (), Shakshi Sharma, Priit Vahter and Rajesh Sharma

Journal of Business Analytics, 2024, vol. 7, issue 4, 273-291

Abstract: This paper investigates which of the core types of innovation can be best predicted based on ’firms’ website data. In particular, we focus on four distinct key standard types of innovations in firms: product, process, organisational, and marketing innovation. Web mining of textual data on the websites of firms from Estonia, combined with the application of artificial intelligence (AI) methods, turned out to be a suitable approach to predict firm-level innovation indicators. The key novel addition to the existing literature is the finding that web mining is more applicable to predicting marketing innovation than the other three core types of innovation. As AI-based models are often black-box in nature, for transparency, we use an explainable AI approach (SHAP - SHapley Additive exPlanations), where we look at the most important words predicting a particular type of innovation. Our models confirm that the marketing innovation indicator from survey data was clearly related to marketing-related terms on the ’firms’ websites. In contrast, the results on the relevant words on websites for other innovation indicators were much less clear. Our analysis concludes that the effectiveness of web-scraping and web-text-based AI approaches in predicting cost-effective, granular and timely firm-level innovation indicators varies according to the type of innovation considered.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/2573234X.2024.2364886 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tjbaxx:v:7:y:2024:i:4:p:273-291

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/tjba20

DOI: 10.1080/2573234X.2024.2364886

Access Statistics for this article

Journal of Business Analytics is currently edited by Dursan Delen

More articles in Journal of Business Analytics from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().