Rutgers University Senate Committee
on Instruction, Curricula, and Advising
Charge S-0318: Examine the various types of online instructional software (courseware) being utilized by departments and units at Rutgers. What are the consequences of using a growing number of different courseware products (sometimes even within an individual course) for faculty, staff and students? Determine whether Rutgers should decide upon and use a standard courseware product.
Course Management Systems
A 2003 Educause study defined a Course Management System as “a software system that is specifically designed and marketed for faculty and students to use in teaching and learning…Most course management systems include course content organization and presentation, communication tools, student assessment tools, gradebook tools, and functions that manage class materials and activities.”[1]
Current Rutgers Course Management Systems
The three primary CMS currently in use at Rutgers are WebCT 4.1 College Edition, Blackboard 6, and eCollege/eCompanion. Several smaller systems with specialized features are also being used.
At Rutgers most courses using CMS are web-enhanced courses rather than web-delivered (i.e. 100% online) courses.
WebCT
· WebCT is currently being used primarily by New Brunswick undergraduate courses, Camden undergraduate courses, and a few Newark courses.
· In the Fall of 2004 there were 354 ‘active’[2] courses with a total enrollment of 17,439[3] students using WebCT. 37 of these were Camden courses.
Strengths: Supports MathML which allows for the use of quiz questions based on formulas with evaluated variables; Supports Unicode for multi-lingual text; Allows the instructor to adjust the release of course content to individual students based on that student’s performance.
Weaknesses: Freeform structure, with multiple, often confusing, course-building options. High learning curve for instructors. Uses a private, internal mail system that can’t be used to communicate with other systems.
In New Brunswick, training and support for WebCT comes primarily from one staff member at the Center for the Advancement of Teaching (CAT); Camden users can also get assistance from the Camden Teaching Excellence Center.
The currently used WebCT 4.1 Campus Edition is reaching the end of its useful life. WebCT Vista, the next major edition of WebCT, would be much more expensive ($90,000 annually as opposed to the approximately $33,000 paid for the Campus Edition) but is a completely redesigned edition that is being tailored for large universities.
Since the next version of Campus Edition will be based on Vista it will have a totally different look and feel and would represent a major change for current users.
eCollege/eCompanion
· eCollege supports RU’s totally online courses.
· In the Fall of 2004 there were approximately 350 students in 27 courses using eCollege.
· eCompanion supports mixed (online and in-person) courses.
· In the fall of 2004 there were approximately 5300 students in 126 sections using eCompanion.
Strengths: Flexible unit-based (e.g., chapter, week, topic), rather than feature-based, course-creation system; WYSIWYG editor Automatic conversion/integration of Word, Excel and PowerPoint documents. An Introduction to the Libraries/ Library Research “course” is available to all Rutgers students. 24/7 Help Desk; virtually no server down time.
Weakness: Few reported weaknesses. Instructor has access to Course (instructional) and Author (design) system views, but not to actual Student view, making it difficult to anticipate common student problems. Microsoft IE 6.0 or higher recommended for users; Netscape and other browsers may not render course features accurately.
eCollege is the course management system which supports entirely online courses university-wide. By policy[4] all Rutgers asynchronous distance learning courses are to use a single, centrally-supported course management system that is administered by Continuous Education and Outreach. This policy affects online courses offered mostly to students at off-campus locations.
As part of a revenue-generating operation, Rutgers eCollege is basically self-supporting. The current annual cost is $150,000, which prepays 1,666 online course transaction fees. Most of this cost is off-set by the online course transaction fee debited to off-campus revenue budgets, resulting in an annual net cost of about $25,000
Since the courses using eCollege are web-delivered rather than just web-enhanced, system robustness, both in terms of individual features and system availability, is crucial. eCollege runs on the provider’s server; the provider offers a high level of server redundancy, and eCollege has a remarkable record of almost 100% server uptime in the last 2 years.[5]
Access to eCompanion is offered to the University community by virtue of the Rutgers eCollege license. eCompanion can be used for any web-enhanced course and offers all eCollege features with the exception of the 24/7 Help Desk; eCompanion help is provided by Continuous Education and Outreach.
Blackboard
· Blackboard has been designated as the official Course Management System for Rutgers-Newark
· In the Fall of 2004, there were 5540 students in 160 Business School courses (60 percent of all business courses; 72 percent of undergraduate business courses) using Blackboard.
· In the Fall of 2004, there were 5064 students in 187 non-Business School courses using Blackboard.
· While the license is primarily Newark-only, Business School courses in New Brunswick as well as New Brunswick Industrial Engineering courses are also using Blackboard.
Strengths: User friendly, easy to learn. “Pool Manager” allows for the creation of a central bank of test questions which can then be exported into “Test Manager” and used for online quizzes and tests.
Weaknesses: Bugs in various components. Whiteboard tends to crash under heavy use (not a major issue for web-enhanced courses).
The Rutgers Business School has been using Blackboard since 2000; in 2004 the system was moved to the Newark server and designated the official CMS for Rutgers Newark.
Newark hired and trained students to act as mentors and work with Economics faculty, the pilot non-Business faculty group, to set up courses in Blackboard. As new departments sign on to Blackboard, faculty are being trained and are receiving assistance in setting up their courses.
In the Fall of 2004, 10,122 students Rutgers-Newark students were in classes using Blackboard.
Outreach and publicity concerning the adoption and use of Blackboard in Newark has been ongoing.
Since Blackboard 6 is now hosted centrally, rosters are now updated daily and final grades can be submitted directly to the Registrar.
Digiclass:
· Digiclass is designed to meet specific needs such as multilingual text and quizzes and shared content across courses.
· Digiclass is being used by FAS language courses and SMLR.
CourseCompass and other publisher systems:
· Textbook publishers are partnering with CMS providers to include online access to their own online tools. Access is then part of the textbook ‘package.’
Sakai
The Sakai program[6] is a collaborative, open source development project that is working towards developing the framework and application tools for an integrated CMS that would also support intra and inter-institutional faculty collaboration and research. Spearheaded by the University of Michigan, Indiana, MIT, and Stanford, the Sakai Project now has about 70 members in its Educational Partners Program. Rutgers, through the office of Instructional and Research Technologies, has made an initial 3-year commitment to Sakai as a member of that Program.
While Sakai has the potential for becoming a focal point for faculty instructional research, its development as a CMS is still relatively primitive and requires significant institutional commitment. Sakai has just released version 2.0.1, which has a greater number of features currently found in standard systems but still has major gaps.
OIT has several staff members working on Sakai CMS development and this Fall about 50 courses are using Sakai as part of a pilot project.
ISSUES:
In Newark, the 2004 decision to adopt Blackboard as the official campus CMS was made at the provostial level and was accompanied by a systematic plan for implementation, training and support. However elsewhere in the University the situation is much more diffuse and unsettled. Rather than being the result of any systematic implementation, the initial adoption of a course management system seems to have a matter of individual faculty initiative or specific departmental need. As a result there are a number of issues including:
· The costs associated with supporting multiple systems
· The relatively low usage of existing systems
· The limited support available for training and course development
· The limited usage of system features
· The need for many students to become familiar with multiple platforms
· The lack of integration between the various course management systems and other administrative systems in the University
Costs:
At present, WebCT is costing the University approximately $33,000 annually; WebCT Vista would increase that cost to about $90,000. The annual cost for Blackboard is currently $55,000. eCollege costs $150,000 annually, although the online course transaction fee reduces that cost to about $25,000. With the exception of eCollege/eCompanion, these costs reflect only licensing and direct implementation. Staff support and training, hardware amortization, and other costs associated with maintaining a CMS are not routinely reported.
It is not clear how much any move toward a single Rutgers course management system would actually save in terms of cost since any significant increase in the usage of a specific system is likely to result in significant increases in the cost of that system. While Sakai currently only requires $10,000 in membership fees for partners, there are significant staff costs involved in terms of initial and ongoing development and maintenance of what essentially becomes a local Rutgers CMS.
Currently, there are approximately 1.5 FTE staff directly or indirectly involved in software development and contributions to the Rutgers Sakai effort.
Usage of Existing Systems
While in the Fall of 2004 there were 354 sections (17,439 students) in New Brunswick and Camden using WebCT, that number represents only about 7 percent of the total sections offered on those campuses. There were another 126 sections (5236 students) using eCompanion.
Despite the fact that Blackboard was only designated as the ‘official’ Newark CMS in 2004, in the Fall of 2004, 10,604 students (33 percent of total enrolled) were in classes using Blackboard. Another 2518 students in 56 courses at the Rutgers Business School in New Brunswick were also using Blackboard.
The initial creation of a web-enhanced course requires a considerable commitment of time and effort on the part a faculty member, especially if that faculty member has not previously used any course management system, or is not familiar with that specific course management system.
The 2003 Educause study of faculty use of course management systems found that “the pressure or persuasion of campus administrators, either at the department chair, dean, or provost level, and the availability of training in the software”[7]
was an important determinant of faculty adoption of a course management system“
Since in New Brunswick and Camden there’s been no particular push to move toward the use of CMS and the support for those developing courses has been limited, it follows that only the most motivated and those with a fairly high level of comfort in the use of technology have taken the initiative to use CMS.
In general, faculty are more concerned with ease and logistics of use of CMS than specific content of individual systems. In terms of organization for use, Blackboard and eCompanion are generally judged superior to the currently used edition of WebCT. However, faculty who have already developed one or more courses using an existing system are likely to be very resistant to being asked to move to another system
Support
The Teaching Excellence Center in Camden offers support in terms of regularly scheduled training sessions for both WebCT and eCollege. In New Brunswick, training and support for WebCT comes primarily from one staff member at the Center for the Advancement of Teaching (CAT). CAT is in no position to support multiple systems or to offer extensive one-on-one support for individual course development.
eCollege and eCompanion training and support is provided regularly to on-campus faculty by Continuous Education and Outreach.
In general, the level of support available to faculty at Rutgers for developing and improving web and web-enhanced courses seems more limited than that available at other institutions of our size.
Usage of System Features
As seems to be generally the case[8], the majority of Rutgers faculty using CMS are not taking advantage of most of the features available to them
A recent survey of some 4400 students at 13 higher education institutions showed that the CMS features least used by faculty were those that students perceived as contributing the most to their learning. [9]
Use of Multiple Platforms
While there are still significant differences between various CMS in terms of approaches in setting up a course, at this point differences in usage from the point of view of student users do not appear particularly significant.
An informal student poll conducted by a Committee member found that students in general don’t care which CMS they have to use.
Feedback that Newark has received from students concerning Blackboard has been primarily focused on issues relating to convenience of use (where they have to login from; how to login) rather than actual utility. While Newark did schedule user-training sessions for Blackboard, no one came.
Students would prefer not to have to log in in multiple places, although they still find that easier than having to use multiple web sites.
Integration with other Administrative Systems
There is a need to better integrate the various course management systems with other administrative systems in the University.
· There are problems with all systems at the beginning of the semester due to constant changes in enrollment due to drop/adds.
· Students who wait to pay bill at last moment are being automatically deregistered even though they may still be attending class.[10]
· Most CMS grade books currently cannot automatically filter into the Registrar’s system and have to be reentered. Newark has resolved this issue in its Blackboard system; OIT is currently negotiating with Administrative Computing to have the local systems accept the data from the other CMS systems.
Multiple systems has generally meant that even problems common to multiple systems have had to be approached and negotiated on an individual system basis.
Other Issues
The University also has an obligation to ensure that vendor-based systems such as those provided by publishers are FERPA-compliant and student privacy rights are being observed.
Movement Towards a Single ‘Official’ CMS
There certainly would seem to be advantages to Rutgers moving toward the use of one primary course management system, especially if that system were to be systematically promoted and supported. Since a decision will have to be made about WebCT--currently the most heavily used Rutgers CMS--in the near future this would seem to be an appropriate time to deal with the issue.
However, which CMS should be chosen is not particularly clear-cut. But since we have already made a significant commitment in terms of staffing for Sakai CMS development, certainly the results of that commitment should be given serious consideration.
This Fall about 50 faculty are participating in a pilot using Sakai as their CMS. Based on feedback from the user group, as well as the assessment of the CAT Fellows, a decision will be made as to the advisability of promoting Sakai as a viable Rutgers CMS.
If a decision were made to support Sakai, the rollout would be modeled on what Newark has done with Blackboard with a campus integration plan, widespread publicity and readily available training and support. Rollout could be as early as January 2006, and
OIT would provide staff support for migrating content from existing classes to a new system.
However we do not see an overwhelming impetus for a “single” Rutgers CMS as opposed to a “primary” Rutgers CMS at this time.
For fully web-delivered courses, system robustness, both in terms of individual features and system availability, is crucial. eCollege, which has been designed specifically for asynchronous distance courses, has proven to have the necessary robustness. If there is a decision to move to Sakai, or another course management system, until there is sufficient evidence that that CMS can appropriately handle 100 percent online courses eCollege should be retained.
Newark has only recently adopted Blackboard as the official Newark course management system in a planned roll-out that should clearly be the model for any future CMS implementation at Rutgers. There would certainly seem to be little point to requiring them to switch at this time.
Rutgers would hardly be alone in its use of multiple CMS platforms. While the 2003 report of the Campus Computing Project found that 71 percent of CMS users at public universities are primarily relying on a single course management system[11], a look at situation at AAU publics [Appendix A] would seem to indicate that this should not be taken to mean that most public universities have moved exclusively to a single course management system.
At 18 of the 33 AAU institutions, even where there is one “officially” supported course management system, other CMS are currently being used by one or more schools, colleges, or departments [12]
Faculty Input
Since the purpose of CMS is to support instruction, faculty input would seem to be core to any selection and implementation process. For the largest possible number of faculty to buy into a new system, both knowledgeable faculty and those not currently using CMS should have the opportunity to specify the features they would deem necessary for any CMS that they would be likely to use.
Recommendations:
1. Rutgers University should move toward the use of one primary course management system.
2. Both knowledgeable faculty and those not currently using course management systems should involved in the selection process, and should be given the opportunity to specify the features they would deem necessary for any CMS that they would be likely to use.
3. OIT should consider the above faculty needs as they work on the development of the Sakai course management system.
4. Implementation of any primary course management system should follow the Newark model of planned and supported integration. Priority should be given to support for faculty training and assistance with course development.
5. Until such time as there is sufficient evidence that the chosen course management system can appropriately handle 100 percent online courses, eCollege should continue to be used for fully web-delivered courses.
6. Until such time as there is a perceived impetus to change, Blackboard should continue to be used as the primary Newark course management system.
Appendix A:
AAU Public Institutions
Use of Course Management Systems
Institution
ANGEL
Blackboard
Desire2Learn
eCollege
WebCT
Local
University of Arizona
Y*
Y
University of California-Berkeley
Y
Y
Y
University of California-Davis
Y*
University of California-Irvine
Y*
University of California-Los Angeles
Y
Y
Y
Y
University of California-San Diego
Y*
University of California-Santa Barbara
Y
Y
University of Colorado-Boulder
Y
Y
Y*
University of Florida
Y
Y*
Y
University of Illinois
Y*
Y
Indiana University
Y*
Iowa State
Y
Y*
University of Iowa
Y*
Y*
University of Kansas
Y*
Y
University of Maryland
Y
Y*
Michigan State
Y*
University of Michigan
Y
Y*
University of Minnesota
Y*
University of Missouri
Y*
Y*
University of Nebraska
Y*
SUNY-Buffalo
Y*
SUNY-Stony Brook
Y*
University of North Carolina
Y*
Y
Ohio State
Y*
University of Oregon
Y*
Penn State
Y*
Y
University of Pittsburgh
Y*
Purdue
Y*
Texas A&M
Y*
University of Texas
Y*
Y
University of Virginia
Y*
Y
University of Washington
Y
Y*
University of Wisconsin
Y*
*Official, centrally-supported CMS
Appendix B:
Committee Process
Between November 2004 and March 2005 the Instruction, Curricula and Advising Committee devoted a number of meetings to the discussion of the use of course management systems at Rutgers.
The Committee met with Gary Gigliotti and Joe Delaney of the Center for the Advancement of Teaching; Richard Novak from Continuous Education; Chuck Hedrick, the University Director for Instructional and Research Technologies; Marie Botticelli, the Director of Newark Computing Services; and Betsy Rowe, Director of Newark Academic Scheduling.
The Committee had presentations on the WebCT and eCollege course management systems and their use at Rutgers, and a report on the use of Blackboard on the Newark Campus.
Several members of the Committee attended the Course Management Systems Workshop held in the Scholarly Communications Center on November 15th which featured presentations on WebCT, eCollege, and Sakai.[13]
The Committee compiled information on the CMS currently being used by AAU Public Institutions. [Appendix A].
A student member of the Committee conducted an informal poll on student use and preferences of CMS.
This report was initially docketed for the April 2005 Senate meeting but was not presented at that meeting due to the extraordinary number of docketed agenda items. Subsequent to that meeting, the committee was given access to the Smith report on Rutgers Course Management Systems <http://rci.rutgers.edu/~itspc/teachlearn/Teaching%20and%20Learning%20Minutes%20031405.pdf> and this report was revised accordingly. In addition, as a decision was made to proceed with the Sakai pilot in the Fall of 2005 the recommendation regarding the pilot was removed from the report.
[1] Morgan, Glenda.
"Faculty Use of Course Management Systems." ECAR Key Findings,
May 2003.
[2] Courses linked to rosters.
[3] Total number registered; 14,655 (84%) were actually participants, that is, they visited the CMS at least once during the semester. The Rutgers CMS course and enrollment numbers used in this report are based on Don Smith’s “Report on Course Management Systems” as presented to the Teaching and Learning Subcommittee of the IT Strategic Planning Committee on March 14, 2005. <http://rci.rutgers.edu/~itspc/teachlearn/Teaching%20and%20Learning%20Minutes%20031405.pdf> As pointed out in that report, there are inconsistencies in what and how data is reported. So these numbers should be considered approximations only.
[4] “Administrative Requirements Associated With the Support
of Revenue-Generating Asynchronous Distance Learning Courses.” Memo from Raphael J. Caprio, Joseph J.
Seneca, Roger J. Dennis, and Norman Samuels to Deans and Directors at Camden,
Newark and New Brunswick. September 14,
1999.
[5] Although for other reasons a student may have not been
able to connect to the system even though the server was up. For example, if the RU network was down in a campus region and an
eCompanion student was in a dorm, a net connection would not be possible to the
eCollege server. However, the student
could access the eCollege servers through alternate ISPs.
[7] Morgan (2003), p.2
[8] The 2003 Educause study found that “use is concentrated on
the content presentation within the CMS.
Faculty members are much slower to adopt the more complex or interactive
parts of the CMS, such as the discussion tools, quiz tools, or
gradebooks….Faculty adopt course management systems principally to manage the
more mundane tasks associated with teaching, especially teaching large classes.”
(Morgan,, p. 2).
[9]Caruso, Judith Borreson.
"ECAR Study of Students and Information Technology, 2004:
Convenience, Connection, and Control."
ECAR Key Findings. September 2004, p.7. http://www.educause.edu/ir/library/pdf/ecar_so/ers/ERS0405/ekf0405.pdf
Top-scoring features
included Sample Exams Online (24.8%);
Online Readings (24.9%); Faculty Feedback on Assignments (32.0%); and the
ability to Share Materials with other Students (38.5%).
[10] eCollege is no longer automatically deregistering students
[11] Angelo, Jean Marie.
“New Lessons in Course Management, ”
University Business September 2004. http://www.universitybusiness.com/page.cfm?p=616
[12] Among these institution, Blackboard and WebCT are still
the most used CMS; each has been selected as the official, centrally-supported
CMS by 10 institutions. The University
of Iowa is currently officially supporting both Blackboard and WebCT. Four institutions have developed their own
systems for use as their primary CMS, while five institutions have recently
moved to either Desire2Learn or ANGEL.
[13] Blackboard was scheduled to participate but pulled out at
the last moment.