Go Back
24th January 2025

Putting AI Chatbots to the Test

AI is becoming a key player in how students gather information online.

When students research universities, the last thing they need is outdated, incorrect, or incomplete information.

AI is becoming a key player in how students gather information online. It pulls details directly from student listing sites, making it essential for universities to keep their profiles accurate and up-to-date.

So, we thought we’d put 3 popular AI chatbots to the test by asking a question a student might typically ask:

Which universities in the UK (outside London) offer the best overall student experience, and why? Please include factors like campus life, extracurriculars, facilities, and academic support. Provide sources for your information.

Recent Updates and Limitations

It’s important to note that ChatGPT’s knowledge was last updated in 2021, meaning it may not reflect the latest trends unless a real-time web search is performed. Gemini (updated in 2024) and CoPilot (updated in October 2023) can also pull real-time information, but if a search misses newer details, they will default to their last update.

Citations and Transparency

It’s also important to note that while these chatbots can provide valuable insights, they often do not explicitly list their information sources. For example, they may not always cite where their knowledge originates, especially when it draws from its pre-existing data. This can lead to gaps in transparency, as users may not always know the exact sources being referenced.

The Importance of Accurate and Up-to-Date Information

As we can see from the results, a varied selection of universities were mentioned, each drawing from different sources—university websites, rankings, student reviews, and more. For example:

  • ChatGPT highlighted Lancaster University, praising its vibrant campus life and modern facilities, citing sources like QS World University Rankings and The Guardian.
  • Gemini included University of Nottingham, focusing on its green campus and thriving student community, referencing Unistats and The Complete University Guide.
  • CoPilot singled out University of Edinburgh for its historic setting and vibrant cultural scene, using sources like Study in the UK.

 

With AI chatbots playing a bigger role in decision-making, it’s more important than ever for universities to keep their listing site profiles accurate and up-to-date. That’s why we created HELM…

Hunterlodge Education Listings Management

Our HELM audit reviews a selection of Higher Education portals, websites, and listing sites. We summarise engagement statistics, where available, and include recommendations from the respective partners. We also dive into your Google Analytics to see performance from your end as well as how well everything is tagged and tracked. Additionally, we review content, user experience (UX), and the accuracy of each profile, offering specific suggestions for improvements. And at just £8,500 for the initial audit and consultancy fee, it’ll come under your procurement threshold and is a great way to invest unspent yearly budgets.

So, stop stressing about your HE listing sites and give us the green light. Contact nick.friend@hunterlodge.co.uk to get started today.

Want to see what we can do for you?

Show Me