Latest Projects

ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience

Authors: Ruiyun Rayna Xu, Yue (Katherine) Feng, and Hailiang Chen

Abstract: The advent of ChatGPT, a large language model-powered chatbot, has prompted questions about its potential implications for traditional search engines. In this study, we investigate the differences in user behavior when employing search engines and chatbot tools for information-seeking tasks. We carry out a randomized online experiment, dividing participants into two groups: one using a ChatGPT-like tool and the other using a Google Search-like tool. Our findings reveal that the ChatGPT group consistently spends less time on all tasks, with no significant difference in overall task performance between the groups. Notably, ChatGPT levels user search performance across different education levels and excels in answering straightforward questions and providing general solutions but falls short in fact-checking tasks. Users perceive ChatGPT’s responses as having higher information quality compared to Google Search, despite displaying a similar level of trust in both tools. Furthermore, participants using ChatGPT report significantly better user experiences in terms of usefulness, enjoyment, and satisfaction, while perceived ease of use remains comparable between the two tools. However, ChatGPT may also lead to overreliance and generate or replicate misinformation, yielding inconsistent results. Our study offers valuable insights for search engine management and highlights opportunities for integrating chatbot technologies into search engine designs.

 

          Download this paper at SSRN

The Impact of ChatGPT Launch on the Demand for Content Generation Services in the Freelancing Market

Authors: Ziqing Kim Yuan and Hailiang Chen

Abstract: The ongoing debate surrounding AI’s impact on the labor market has been fueled by the rise of generative AI. While some argue that it may displace jobs, others suggest it could create new opportunities and improve productivity. This study investigates the impact of the ChatGPT launch on the demand for human content generation services, using a difference-in-differences approach and data from the online labor marketplace Fiverr. Our investigation reveals a significant decrease in demand for human content generation services following the ChatGPT launch. Moreover, we explore the heterogeneity of the impact, identifying differential effects across service types, human writers’ education levels, and service pricing. Notably, we observe a significant increase in demand for idea planning services, while demand for text reviewing services and services provided by human writers with a PhD degree remains unaffected by the launch of ChatGPT. These results contribute to the ongoing debate on the impact of generative AI on the labor market and offer practical guidance for human writers navigating this evolving AI-driven landscape.

 

          Download this paper at SSRN

SocioLink: Leveraging Relational Information in Knowledge Graphs for Startup Recommendations

Journal of Management Information Systems 40(2) 655-682. 2023.

Authors: Ruiyun Rayna Xu, Hailiang Chen, and J. Leon Zhao

Abstract: While venture capital firms are increasingly relying on recommendation models in investment decisions, existing startup recommendation models fail to consider the uniqueness of venture capital context, including two-sided matching between investing and investee firms and a lack of information disclosure requirements on startups. Following the design science research paradigm and guided by the proximity principle from social psychology, we develop a novel framework called SocioLink by depicting and analyzing various relations in a knowledge graph via machine learning. Our experimental results show that SocioLink significantly outperforms state-of-the-art startup recommendation methods in both accuracy and quality. This improvement is driven by not only the inclusion of social relations but also the superiority of modelling relations via knowledge graph. We also develop a web-based prototype to demonstrate explainable artificial intelligence. This work contributes to the FinTech literature by adding an innovative design artifact—SocioLink—for decision support in the investment context.

 

     Download this paper at SSRN

Listening in on investors' thoughts and conversations

Journal of Financial Economics 145(2) 426-444. 2022.

Authors: Hailiang Chen and Byoung-Hyoun Hwang

Abstract: A large literature in neuroscience and social psychology shows that humans are wired to be meticulous about how they are perceived by others. We propose that impression-management considerations also end up guiding the content that investors transmit via word-of-mouth. We analyze server-log data from one of the biggest investment-related websites in the United States, as well as experimental data. Consistent with our proposition, we find that investors more frequently share articles that are more suitable for impression management, even when such articles less accurately predict returns. Additional analyses suggest that high levels of sharing can lead to overpricing.

 

          Download this paper at SSRN

Fake News, Investor Attention, and Market Reaction

Information Systems Research 32(1) 35-52. 2021.

Authors: Jonathan Clarke, Hailiang Chen, Ding Du, and Yu Jeffrey Hu

Abstract: Does fake news in financial markets attract more investor attention and have a significant impact on stock prices? We use the SEC crackdown of stock promotion schemes in April 2017 to examine investor attention and the stock price reaction to fake news articles. Using data from Seeking Alpha, we find that fake news stories generate significantly more attention than a control sample of legitimate articles. We find no evidence that article commenters can detect fake news. Seeking Alpha editors have only modest ability to detect fake news. The broader stock market appears to price fake news correctly. The stock price reaction to the release of fake news is not significantly different than a matched control sample over short and longer-term windows. We conclude by presenting a machine learning algorithm that is successful in identifying fake news articles.

 

Download this paper at SSRN

Signal or Noise in Social Media Discussions: The Role of Network Cohesion in Predicting the Bitcoin Market

Journal of Management Information Systems 37(4) 933-956. 2020.

Authors: Peng Xie, Hailiang Chen, and Yu Jeffrey Hu

Abstract: Prior studies have shown that social media discussions can be helpful in predicting price movements in financial markets. With the increasingly large amount of social media data, how to effectively distinguish value-relevant information from noise remains an important question. We study this question by investigating the role of network cohesion in the relationship between social media sentiment and price changes in the Bitcoin market. As network cohesion is associated with information correlation within the discussion network, we hypothesize that less cohesive social media discussion networks are better at predicting the next-day returns than more cohesive networks. Both regression analyses and trading simulations based on data collected from Bitcointalk.org confirm our hypothesis. Our findings enrich the literature on the role of social media in financial markets and provide actionable insights for investors to trade based on social media signals.

 

Download this paper at SSRN