Back to Blog
Web App Development Infrastructure History

SearchNEU: The Modern Alternative to Northeastern's Course Catalog

Jun 23, 2024 6 min read

by Pranav Phadke and Ryan Hughes

Navigating Northeastern University's course catalog can be daunting, especially with the limitations of the Banner system. This article explores the challenges associated with Banner, how Ryan Hughes (our founder) tackled these issues with the creation of SearchNEU, and how this innovative app has evolved with the contributions of Northeastern students. Search NEU started in 2017, grew to 50k+ users by 2019, and has been a staple at NEU ever since.

Difficulties using Northeastern's official Course Catalog (Banner)

Banner, the course catalog application developed by Ellucian, is notoriously cumbersome. Accessing it requires logging in through Northeastern's student hub, and it lacks mobile responsiveness, limiting searches to a computer. The search process itself is tedious. For instance, to find "Fundamentals of Computer Science 1," you need to know its course number (2500) and subject (Computer Science) to find it in Banner.

Banner's user interface is antiquated and resembles a pre-2000s spreadsheet. Being an outsourced application, the Northeastern community cannot make feature requests to improve the user experience or address privacy concerns. Additionally, the site is prone to outages, leaving students without a reliable, Northeastern-provided alternative for course information.

Another major shortcoming with Banner is its inability to notify students when seats open up in fully booked classes. This makes course registration frustrating, as students must constantly check for openings, often missing out due to the unavailability of timely alerts.

SearchNEU: A Better Solution

SearchNEU, often referred to as "Search", is a search engine designed for easier navigation of class and professor information. It is mobile-responsive, requires no login, and offers notifications when seats in previously full sections become available.

The origin story

The story begins in 2016 when Ryan Hughes and some friends wanted to register for specific classes for the next semester. However, by the time they could register, those classes were fully booked. When Ryan asked the registrar for advice, they recommended constantly checking NEU's course registration website to see if a seat opened up. Instead of manually checking the site every day, Ryan developed a script to automatically check and send notifications if it found a seat. This initial script was a success and widely used by many Northeastern students.

The first version of Search NEU

Ryan then started addressing various usability issues, which required pulling significantly more data from Northeastern's website. Since the website lacked an API, Ryan built web scrapers that created objects representing semesters, courses, and sections. The entire course catalog was scraped every 24 hours, and a smaller script checked for open seats every five minutes.

This initial version of the app was built on a single t3.medium on AWS. Cloudflare was used as a CDN, Travis CI was used for CI and notifications were sent to users with Facebook Messenger's API. The data itself was stored in Elastic Lunr.js on the backend and user data was stored in Firebase. Google analytics and Amplitude were used for analytics.

This first version of SearchNEU was a success and quickly spread throughout Northeastern.

SearchNEU's Evolution

A few years later, in 2019, Ryan was graduating college and was looking for a group of students to continue the project. Around the same time, a group of Northeastern students (Ryan Drew, Da-Jin Chu, Mitch Gamburg) founded a student-led software consultancy called Sandbox. Their goal was to develop industry standard software for Northeastern researchers, professors, and the overall student community.

Sandbox proved to be an excellent organization to pass SearchNEU to because the founding team shared the same goals and had the necessary expertise. They adopted SearchNEU as their cornerstone project and initiated several other projects.

Timeline of SearchNEU over the last 5 years

2019

  • Migrated frontend to Vercel
  • Migrated from ElasticLunr.js to ElasticSearch
  • Migrated from Firebase to PostgreSQL
  • Built a GraphQL API for GraduateNU to utilize course-catalog data
  • Transitioned infrastructure to Kubernetes and Docker on AWS

2020

  • Updated scrapers for new Banner versions
  • Added support for CPS and LAW catalogs
  • The design of Search NEU was re-done

2021

  • Enhanced scraper support for additional colleges
  • Transitioned from Facebook Messenger to Twilio for notifications

2022

  • Added phrase searching to elastic search querying
  • Big focus on improving Mobile responsiveness
  • Rewrote backend code in TypeScript

2023

  • Updated to Node.js version 18 as required by Vercel
  • Updated Next.js, React.js, and other dependencies
  • Updated backend documentation
  • Created a DumpProcessor for mid-semester section additions
  • Implemented an API-error page
  • Replaced deprecated requests library

2024

  • Created a subscription page to manage all section notifications
  • Updated the sign-in flow for notifications
  • Mark past terms as inactive

What users are saying

"I missed the summer class registration deadline, but SearchNEU's SMS reminders helped me secure the classes I needed to graduate on time." — Northeastern Student
"SearchNEU's convenience, speed, and better UI make it perfect. No need for Duo authentication to look for classes." — Northeastern Student
"As an associate dean, I frequently use SearchNEU for enrollment information. It's easier to access than Banner, reliable, and presents information clearly." — Associate Dean
"I'm a CS faculty and assistant director at NU Vancouver. SearchNEU is essential for my workflow." — CS Faculty

Conclusion

SearchNEU has proven to be a game-changer for the Northeastern community, offering a modern, user-friendly alternative to the cumbersome Banner system. Through the dedication and innovation of students and teams like Sandbox, SearchNEU has continuously evolved to meet the needs of its users. Its journey from a simple script to a robust application with thousands of users underscores the impact of student-driven solutions in enhancing campus life.