Open-Rate Controlled Experiment in E-Mail Marketing Campaigns

Antun Biloš (), Davorin Turkalj () and Ivan Kelić ()
Additional contact information
Antun Biloš: Faculty of Economics, University of Osijek
Davorin Turkalj: Faculty of Economics, University of Osijek
Ivan Kelić: Faculty of Economics, University of Osijek

Tržište/Market, 2016, vol. 28, issue 1, 93-109

Abstract: Purpose – The main purpose of this paper is to test the controlled experiment (A/B split) methodology in B2C oriented e-mail marketing campaigns. Design/Methodology/Approach – E-mail marketing techniques have been a substantial part of e-marketing methodology since the early Internet days of the mid-1990s. From the very beginning of Internet utilization for business purposes, e-mail was one of the most widely used communication techniques in B2B and B2C markets alike. Due to high volumes of spamming and progression of online communication clutter, some practitioners began to question the usability of e-mail as a marketing communication channel, while others embarked on working on improving the message itself. Efforts were invested into improving message quality, as well as into better understanding user expectations. One of the most commonly used techniques to test specific e-mail message elements is the controlled experiment. Findings and implications – This paper explores several types of controlled experiments in a specific Croatian B2C market. Tests were run to determine subscriber behavior towards several newsletter components, including sending time, sending day, sender’s name, and subject line. Open and click rates for tested campaigns, and several other metrics were investigated using MailChimp software. An N − 1 two-proportion test using an adjusted Wald confidence interval around the difference in the proportions was used for comparing the open-rate measure in the controlled experiments between subjects. Limitation – Controlled experiments (A/B split tests) showed a lot of potential as a way of measuring behavior and preferences of subscribers, although several apparent limitations (the data-set scope, comparability issues) indicated a clear need for standardization on a managerial and scientific level. Originality – This paper provides an up-to-date e-mail marketing effectiveness literature review, describes and tests the methodology and metrics for e-mail campaigns measurement, and suggests several important guidelines for further research.

Keywords: controlled experiment; A/B split test; e-mail marketing campaigns; newsletter (search for similar items in EconPapers)
Date: 2016
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://hrcak.srce.hr/file/236637 (application/pdf)
None

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zag:market:v:28:y:2016:i:1:p:93-109

Ordering information: This journal article can be ordered from
Tržište/Market, Faculty of Economics and Business, Trg J. F. Kennedy 6, 10000 Zagreb, Croatia.
http://www.efzg.unizg.hr/trziste

Access Statistics for this article

Tržište/Market is currently edited by Đurđana Ozretić Došen

More articles in Tržište/Market from Faculty of Economics and Business, University of Zagreb Contact information at EDIRC.
Bibliographic data for series maintained by Tanja Komarac ( this e-mail address is bad, please contact ).