Any Macintosh, PC, or UNIX system that is connected to
the network and has a local disk.
The Software Requirements are divided into two sections --- General Description and Specific Requirements.
The General Description provides the information necessary to understand the specific requirements for the complete system. The General Description is divided into five main sections.
- The Project Perspective : How the backup and archiving system relates to the larger CMU central computing environment.
- The Project Functions : What services the backup and archiving system will provide.
- User Characteristics: What type of people will interact with the system at various levels.
- General Constraints: Any items that will limit how the system is designed
- Assumptions and Dependencies: Factors that, if changed, will affect the rest of the design.
The Specific Requirements section describes all of the features necessary in the complete backup and archiving system. Each feature described in this section is prioritized as mandatory, highly desirable, or desirable.
If you have any questions about the requirements, or if you are interested in providing or developing part or all of the Carnegie Mellon Backup and Archiving System, please contact:
Mark Held
markh+@cmu.edu
(412) 268-5158
Wallace Colyer
wally+@cmu.edu
(412) 268-6497
Alex Margita
am3f+@andrew.cmu.edu
(412) 268-6688
Fax to
(412) 268-4987
Carnegie Mellon Computing Services currently provides the campus community with a network-based service called Andrew.
Andrew service includes:
- A network with Ethernet, Token Ring, AppleTalk, and dialup
asynchronous and SLIP connections
- AFS file servers. There are 22 file servers, each containing four 800 MB disks. As of November 1992, total storage capacity is almost 63 gigabytes. Each year, the total storage capacity is increased by roughly ten percent.
- Printing
- Electronic mail and bulletin boards
- Kerberos authentication
- Remote terminal interfaces
- A backup system for the file servers
Andrew is used by over 8,000 students, faculty, and staff. Approximately 1000 UNIX workstations, 500 PC's, and 2000 Macintoshes are hooked up to the network though Ethernet, Token Ring, or AppleTalk connections and use Andrew Services. There are also AppleShare and Novell NetWare File Servers accessible through the network.
There are additional computing facilities for individual departments or research organizations. These facilities will not be included in the new backup and archiving service.
Computers that access the campus network through dial-up connections are not to be included in the new backup and archiving service.
Currently, the only data that is backed up centrally is the data stored on the AFS file servers. In order to back up this data, three DECstation 2100s and a DECstation 3100, all with Storage Expansion Units, are used as backup servers. Each machine acts as a storage buffer between the Andrew fileservers and the 8mm tape drives. After the Andrew volumes are copied onto the backup servers, the servers alert the computer operators to mount the tapes needed for the backup. Operators mount tapes on the backup servers throughout the day. Files are backed up daily, weekly, and at the end of the fall, spring, and summer semesters.
Backups are copied onto 8mm tapes. The tape library contains 9-track tapes from our previous backup system that must be accessed for file restores.
If users want a file restored to them, they must contact Computer Operations and provide detailed information on the file to be restored. An operator then searches for the file in a database, mounts the appropriate tape, and restores the data to the user.
The existing system has the following problems:
- Each backup server handles a unique data set and tape set, making it difficult to share resources such as disk space, tapes, and tape drives.
- The hardware does not stand up to hard, constant use.
- It is difficult for computer operators to restore files to users.
- The system does not give any feedback to users.
- The backup software is home-brewed and difficult to support.
At this time, Computer Services does not provide central backup services for the local disks of workstations hooked up to the network. Users must provide their own backups in one of several ways.
- Users can make their own backup copies and archives on personal floppy disks, additional hard disks, or tape drives.
- PC users can migrate their data to their AFS file space using PCserver software. This software lets users treat their AFS file space as a virtual network disk. The data will then be backed up through the AFS central backups.
- Users can migrate their data to Novell NetWare File Servers, which are typically backed up centrally using ARCserve by Cheyenne, Inc.
- Users can migrate their data to AppleShare servers, which are typically backed up on a regular basis
These options have the following problems:
- The hardware, software and backup media necessary to perform backups are too expensive for most individuals and organizations on campus.
- AFS disk space is limited, and most individuals do not have enough quota to store large amounts of data.
- Not all users have access to Novell and AppleShare servers that permit copying and storing data.
- Not all Novell and AppleShare servers are backed up regularly.
Computing Services is planning a new centralized network service offering
distributed computing for UNIX, Macintosh, and IBM PC workstations.
This system will include:
- network connections and dialups
- DFS file servers
- Printing
- Electronic mail and bulletin boards
- Kerberos authentication
- License service
- Remote terminal interfaces
- A central backup and archiving system for AFS/DFS, AppleShare, and Novell NetWare File Servers as well as for all UNIX, Macintosh, and IBM PC workstations on the network.
The project's short-term goal is to provide affordable, worry-free backup and archiving services to Macintosh, PC, and UNIX workstations through one or more commercial backup systems as soon as possible.
The long-term goal of the Backup and Archiving System is to provide reliable backups of the DFS Servers at Carnegie Mellon by January of 1994.
The central computing Backup and Archiving System must serve two main
functions:
Backups are used to restore all or part of the central computing environment in case of a disaster or an accident.
- In case of a disaster, where one or more disks of the central computing computers have been destroyed, the backup system must be able to restore the most recent backup image of the data on that disk.
- In case of an accident, where a user loses part or all of their data, the backup system must be able to restore individual files, directories, or volumes from the most recent backup image to replace the lost data.
Archives are used to store data on the most appropriate medium in terms of cost and accessibility.
- Automatic archiving moves all files that the user has not touched for a set period of time to the University's central storage facilities. The user can then retrieve these files when they are needed.
- User-initiated archiving allows the user to move specific files to the University's central storage facilities on an individual basis.
The central backup and archiving services can be provided by one integrated system or by several smaller subsystems. The Requirements presented in this document are for the backup and archiving system as a whole. We will consider any independent subsystem that provides a subset of these requirements.
A user is any individual who has data backed up by the system. This includes every student, faculty, and staff member with an account on the AFS/DFS file servers or a personal computer hooked up to the network.
Users range from novice to expert. All incoming students take a Computing Skills Workshop to introduce them to the basics of using the campus Macintoshes and UNIX workstations. There are no computer training requirements for faculty and staff.
Computer Operators provide twenty-four hour supervision and maintenance for central computing facilities. These duties include monitoring system performance, initiating backups, providing backup media, and restoring data to users.
Computer Operators undergo intensive training on the systems they care for.
Administrators are in charge of changing parameters and system settings to the most appropriate level for the current environment. They also track the system's resources and monitor the system's performance.
Administrators have expert knowledge and a thorough understanding of the central computing environment, as well as the backup system.
Software Developers and Maintainers are system programmers who are responsible for maintaining the integrity of the backup system's software. They may also modify the behavior of the backup system or fix problems with the system.
Software Developers and Maintainers have expert knowledge and a thorough understanding of the central computing environment, as well as the backup system. These individuals are typically able to rebuild the backup system from source files.
Computing Services cannot physically access all of the workstations on campus.
The Carnegie Mellon network is available to a wide variety of users, including administrators and students. Each person's or organization's data must be
protected from action or intrusion by naive or malicious users.
Many workstations on campus are publicly available to any student, faculty, or staff member. Other workstations are private, and can only be operated by specific users.
The entire environment includes AFS/DFS file servers, 1000 UNIX machines, 2000 Macintoshes, and 500 IBM PCs. Most of the UNIX, Macintosh, and PC workstations have access to central AFS/DFS, AppleShare, and Novell NetWare file servers. The amount of data created and stored by users is growing by 25% every year.
Because workstations are physically unaccessible, the system must be network-based.
Because some users of the network will be naive or malicious, the backup system must provide data protection --- that is, prevent unauthorized network access to the system's data or functions.
Because the computing environment is continually expanding, the backup system must be scalable --- that is, able to adjust easily to system growth.
Because the system must be network-based and provide data protection, an authentication process must verify the identity of entities making requests.
- Individual workstations will have to be authenticated for automatic backups.
- Users will have to be authenticated for individual archiving services.
The backup system for AFS/DFS servers assumes that DFS will exist in the near future.
Features described in the Specific Requirements fall into three different categories --- Mandatory, Highly Desirable, and Desirable.
- Features that are mandatory must be included in the central computing Backup and Archiving System.
- Features that are highly desirable should be included in the central computing Backup and Archiving System.
- Features that are desirable may be included in the central computing Backup and Archiving System if they fall within our resource limitations.
These are requirements for the entire backup and archiving system. We will consider any independent subsystem that provides a subset of these requirements.
It is mandatory that the backup system:
- run over a Local Area Network
- be compatible with Ethernet, AppleTalk, and IBM Token Ring
- be able to backup AFS/DFS, AppleShare, and Novell file servers.
- be able to backup UNIX, Macintosh, and PC/DOS workstations.
It is highly desirable that the backup system:
- support all of the listed client systems in one package.
- meet network protocols --- TCP/IP, AppleTalk, and NetWare.
It is desirable that the backup system:
- provide multiple choices for the hardware platform of the backup servers.
It is mandatory that the backup system:
- support backup servers pulling data from workstations using an agent.
It is highly desirable that the backup system:
- support workstations pushing data to backup servers.
It is mandatory that the backup system:
- support over 1,000 UNIX, 500 PC, and 1000 Macintosh client workstations.
- support multiple backup servers.
- backup an unlimited number of files of unlimited file size.
- be able to maintain backups of over 100 gigabytes of data.
It is highly desirable that the backup system:
- support multiple backup devices for each backup server.
- appear available to perform tasks and respond to requests from users and administrators 24 hours a day.
- perform all the actions necessary for a user to start a service immediately.
- respond interactively and provide feedback to the user immediately.
It is mandatory that the backup system:
- catalog file histories and maintain a database of all the files backed up.
- allow users to archive, or store individual files on storage media.
It is highly desirable that the backup system:
- distribute and duplicate the file history catalog on multiple tape servers.
- support the migration, or disk grooming, of little used files to cheaper media.
It is mandatory that the backup system:
- have configurable scheduling controls for automatic backups, so we can change the time backups are run.
- store configuration options centrally.
- have an automatic backup process that handles backup scheduling and only requires a computer operator to mount the appropriate storage media.
- allow computer operators to perform unscheduled backups.
- be able to perform full and incremental backups.
- provide the option for users to schedule automatic, unattended requests for backups of their local data from their personal workstations.
- deal appropriately with machines that are inaccessible or turned off.
- be able to filter out files that should not be backed up. Users should be able to exclude files that they do not want to have backed up, and Administrators should be able to exclude files that they do not want to back up.
It is desirable that the backup system:
- check the integrity of the files it is backing up and log any problems.
It is mandatory that the backup system:
- provide a file history catalog that users may browse.
- restore files in their native format. Therefore, a MacWrite document returns to the users as a MacWrite document.
- be able to backup and restore the backup system itself for the purpose of rebuilding the backup system from a previous state.
- be able to recover an entire workstation or file server.
- determine automatically which tapes are required for recovery.
It is highly desirable that the backup system:
- restore files to their original location in the user's file space.
- restore data promptly as soon as the system has been provided with the proper piece of media.
It is desirable that the backup system:
- be able to restore files from one platform to another platform.
It is mandatory that the backup system:
- be able to automatically make a second, or shadow, copy of backups that can be stored off-site.
It is highly desirable that the backup system:
- use an industry-standard, non-proprietary format for the structure data stored in on backup media.
- write "sanity labels" on the backup media that allow the system to verify that the proper piece of backup media has been mounted.
- be able to check tapes already written to make sure the data is still OK.
- perform services without the assistance of computer operators, except to mount media if an auto-mounter is not used.
It is desirable that the backup system:
- be able to compress data during network transfer and/or the media write.
It is mandatory that the backup system:
- allow active monitoring of the backup system so computer operators can watch the status of all processes.
- provide automatic reports of its successes and failures.
- provide job status and feedback reports to users.
- provide statistics that can be used for billing.
- monitor how often a specific piece of media is used and the number of device errors with that piece of media and then recommend when that media should be expired and its data transferred to another piece of media.
It is highly desirable that the backup system:
- monitor how often any locally attached devices are used and any device errors that occur and then recommend service and preventive maintenance for the device.
- provide an interface that retrieves resource accounting information on all jobs and requests processed (whether successful or failed). This information should include any statistics about local resources used such as elapsed and CPU time, I/O counts to media storage devices, and remote file location, size, name, and description.
It is mandatory that the backup system:
- provide security beyond authentication of network addresses.
- provide access controls to various functions of the backup system including but not limited to: service requests, statistical reports, and administrative controls or settings.
It is highly desirable that the backup system:
- be able to determine the owner of the backed up files and verify user identity for restores.
- use Kerberos authentication.
It is desirable that the backup system:
- be able to encrypt data transferred over the network.
It is mandatory that the backup system:
- be able to operate 24 hours a day.
- provide modular hardware components.
- use a standard hardware interface such as SCSI-2 bus to connect the backup server to the backup devices.
It is highly desirable that the backup system:
- provide access to schematics and diagnostics for the hardware. These would assist in-house maintenance and reduce downtime.
- use a jukebox or robot-controlled device to minimize human interaction with hardware.
- be able to use 8mm tapes.
It is desirable that the backup system:
- be able to use 9-track tapes.
- be able to use Optical devices.
It is mandatory that the backup system:
It is highly desirable that the backup system:
- have good programming interface which allows for expansion to future devices and client platforms.
- provide access to the source code of any vendor-supplied products to Software Maintainers.
It is mandatory that the backup system:
- be as simple and "user-friendly" as possible.
- be consistent with the user interfaces on other platforms if part of an integrated backup and archiving system.
It is highly desirable that the backup system:
- express the look and feel of the native user interface for each platform and
follow all human interface guidelines for that platform.
- be compatible with X.11 R4 when used on UNIX workstations.
Table of Contents 
Next Chapter 
Previous Chapter 