Overview
Mailing lists, also known as distribution lists, are an easy way to communicate via email: an email sent to the list will go out to all the subscribers of that list. The Stanford mailing service is based on the popular open-source application Mailman. The service currently hosts over 29,000 mailing lists, which serve all of the campus community as well as outside subscribers. The lists serve as a communication platform for a variety of official and casual purposes. The Mailman software provides a powerful and flexible platform for list owners and subscribers, allowing them to manage their lists and personal configurations on a per-list basis.
Current State
IT Services provides list hosting for departments, groups, classes, and personal use. Currently, we provide and support:
- Server software upgrades and security patches.
- Mailman software integration with WebAuth and archive searching.
- Nightly backup of the list archive data to the failover server.
- Hourly backup of the list configuration data to the failover server.
- Tape backup of list data on the failover server (as not to impact the production server).
- Web tools for requesting new lists and for Help Desk vetting of list requests.
- A simplified web interface for list administration.
- Remctl command line tools for lists administration by the Help Desk.
- 24/7 outage and list delay coverage.
- Email notification to customers of changes and maintenance.
- Mailman packages with Stanford patches managed by git repository.
- Workgroup linkage to lists.
- Mail tagged with Secure: is now rejected by mailing list due to the limitations of secure email software.
Vision
Due to the enormous number of lists, the mailing list service has experienced scalability issues in the past few years. At best, only a third to half of the 25,000 lists are currently active. There is currently no expiration policy for inactive lists. Thus, a policy rewrite, along with better list cleanup tools, are the top priorities for the health of the list service.
Through modifications to email routing, many of the scalability and performance issues have been resolved, though misuses of the list service can cause occasional service degradation. Examples of misuse are use of the lists for automated server warnings, and for bounce handling of bulk mail. IT Services is working with clients to cut down on such abuses. With the tweaks to the mail routing in place, the list service should be able to survive on its current hardware for two to three more years. IT Services has already scheduled migration to a larger storage volume using the current storage solution for Mailman. Improvements to the scalability and usability of the product will depend on actively working with the existing development effort. Resources need to be allocated to evaluate where the 3.0 release stands and what opportunities exist for Stanford to influence the direction of the product.
The mailing list service is expected to continue to be an integral part of IT Services' email offerings. The number of configuration options and the way they are scattered around Mailman's own interface frequently leads to misconfigured lists and resulting HelpSU requests. The simplified interface that IT Services developed initially for its rollout of Mailman is not sufficient to mitigate these problems. A revision would reduce the overall support cost of this service and likely improve its stability.
Goals
- A lifecycle management process that will automatically find and remove inactive lists as well as lists that no longer have Stanford owners.
- Maintain a connection with upstream development work, especially on Mailman version 3.0, in order to have Stanford-generated patches incorporated and to guide development, so that the next version will fulfill Stanford's future needs.
- A more flexible storage solution that will allow an incremental increase in the amount of storage as the list archives grow in size.
- Better automatic list membership population tools, based on CSV (Comma-Separated Values) files.
- Better metrics around the use of this service, such as the amount of mail flowing through the list servers, breakdown of the number of subscribers per list (including Stanford subscribers versus non-Stanford subscribers), and spam and virus trends.
- An improved, simplified user interface, preferably by working to get it adopted into the Mailman distribution.
- Full integration of the web interface with the list (allowing posting as well as reading).
- Member profiles (which requires global member profiles rather than per list member profiles).
- Basic content management (simple web pages, photos, uploading/downloading of files).
Roadmap
- Revise list creation and maintenance policies
- Automate more functions to incrementally improve the service.
- Write tools to measure metrics such as breakdown of lists by membership size, archive size, average mail traffic, top lists, and so forth.
- Complete the migration to a larger storage volume for archive data.
- Work with Administrative System to better integrate the list service with Workgroup Manager and PeopleSoft class lists.
- Remove inactive lists and non-Stanford affiliated lists.
- Help develop a more comprehensive list usage policy to cut down on abuses of the list service.
- Determine needs for a revised, simplified interface and evaluate options to deliver it.
- Meet to support secure email through mailing lists.
- The new version of Mailman version 3.0 is currently in alpha. IT Services needs to be more engaged with the development effort on that version so it will have the features Stanford needs.
- Continue to look at options such as clustering to ensure the scalability of the service.
Measures of success
- A clearly defined list creation and maintenance policy.
- A policy and procedure that would enable IT Services to initiate the deletion of inactive lists. Currently, the list deletion process is user-initiated.
- A list service remains stable with little or no user visible delays in mailing list traffic.
- Active involvement in Mailman 3.0 development to ensure that the list service will continue to move forward and incorporate the best available technologies in the future.
- Net reduction in frequency of HelpSU requests relating to Mailman.
