Open Source Learning Tools
Charles Kerns
Education Technology Manager
Academic Computing,
Stanford University
The Presentation
Learning tools and open source
CourseWork, Stanford’s CMS
The Assignment and Assessment
Manager, a new, modular CourseWork
What is Open Source?
Licensed software that can be freely
License allows users to modify and
adapt to their own needs
May have restrictions on redistribution
Community of developers and users
Open Source Programs
that We Already Know and Love
In fact, a lot of the
internet runs on open
source software!
The Vision
Institutions sharing open source tools
Basic open source frameworks that accept
tools and integrate them with:
Campus SIS, Portal, Authentication System, File
Servers, Calendar, etc.
Modular, interoperable, open source tools
Rich set of open source tools to support:
discussion, quizzes, learning communities,
problem sets, project-based learning, syllabi,etc.
The Need-Control of the
Campus Learning Tools
to integrate learning tools into campus infrastructure
SIS, file servers, authentication, email, portal, calendar
to modify tools to fit pedagogical methods of faculty
to add new tools requested by faculty
to develop content with faculty using learning tools.
to always retrieve faculty content - content is NEVER
locked in proprietary formats
to have tool continue even if original developer “dies”
Why an Open Source CMS?
Port software to your environment: no longer
restricted by vendor requirements
Modify and expand software to meet your needs
and integrate with existing campus systems
Easy to integrate features, customizations, and
fixes developed by other users
Work with a common technical framework to
share reusable learning objects
More Reasons Why
Open standards allow interoperability of
applications and services
Non-proprietary data formats mean
easier conversion to or from the system
Reduced acquisition costs: no need to
pay for evaluation software, and can
take as long as you need
You don’t have to pay for upgrades!
But that doesn’t mean you get
something for nothing….
Must have or develop internal expertise to manage
the system and development tools
Extensive customization may result in development
and support costs equal to or greater than the cost
of commercial software….
…but at least you have control: you’re no longer
dependent on vendors for upgrades and
Recent History of Learning Tools
Hundreds of Course Management Systems and
Learning Tools with many as Open Source
Thelma Looms Listing at GWU
 160 with a few “free” from 98 to 02
 235 plates-formes e-formation, plates-formes e-learning
with 28 Open Source - current
EduTools Listing
 32 with 10 Open Source - current
Some Open Source CMS’s
Open Learning Management System - Dept of
Psych-Utah – Basic System
FLE3 Future Learning Environment –Helsinki
– Collaborative Knowledge Bldg
Moodle by Martin Dougiamas, Australia –
Week-by-week Courses
Some more Open Source CMS’s
CHEF –a full portal with research and
instructional support system
OnCourse - Indiana’s reworked CMS
CourseWork – Stanford’s 80% solution.
Supports lectures and seminars.
The History - Research and
Non-Education-Specific Apps
Collaborative Knowledge Building
Virtual Worlds for Chat
Structured Discussion
Content mixed with Tools
The History - Commercial and
Open Source Learning Tools
No standards
No modularity
Little collaboration among developers
Much redundancy
Some unique applications
collaborative knowledge building,
structured discussion,
problem-based learning
The History - Commercial and
Open Source Learning Tools
All are vertical apps leading to:
Much effort for Sys Admins
Much time for faculty
Many separate apps to maintain and integrate
Multiple student lists, Separate systems for
assessment, difficult to relate components
Much confusion for students
 Multiple sites, multiple passwords, no
integration between activities
What’s Needed
Standards – OKI, IMS
Organization for tech support & distribution
A Merlot++ for infrastructure tools??
User community
Many users helping each other
Development community
Several developers of tools using same framework
What’s Happening-Standards
OKI Collaboration of many universities,
funded by Mellon Foundation.
Content Packaging, QTI, etc. DTD’s
What’s Happening - Open
Knowledge Initiative (OKI)
Collaboration of many universities, funded by
Mellon Foundation.
Define open architectural specifications to support the
development of educational software.
Provide a modular and extensible development
platform for building both traditional and innovative
educational applications while helping institutions
leverage existing infrastructure.
More information:
What’s Happening - OKI
Develop standard APIs for:
Common Services
 Authentication
 Authorization
 Database Connectivity
 Filing
 Logging
Class Info (student lists)
Assessment and Grades
What’s Happening - User
Groups & Distribution Support
Merlot (Learning Object Sharing)
Content and Tools
LON CAPA and WebWork have user groups
CourseWork Evaluation at Denison &
other colleges
CHEF Workshops
Hewlett/Mellon looking at distribution
What’s Happening - Developers
starting to collaborate
First goal is to use each others’ tools in proofof-concept demonstrations
OKI users working together (It takes more
than APIs)
Umich CHEF
Indiana OnCourse
Stanford CourseWork
MIT Stellar
Tufts Concept Mapping Tool
Evaluation for Open Source Release
5 Universities/colleges (already selected) are
setting up local server and testing.
They will report on features, admin issues,
compatibility with their campus systems.
Two have servers running with little assistance;
third needed help in setting up Tomcat
Denison University is evaluating CourseWork for
reporting to CLAC
Grant proposal submitted to Mellon Foundation
for support for outreach to liberal arts colleges
Changes will be made in open source version
based on evaluation.
Source is available with free download
Can modify or adapt without notifying
Can redistribute original or modified if
do not charge
Stanford holds copyright on CW
Cannot sell CW
Questions on State of Open
Source Tools . . .
Tech Docs
Design Goals/Methods
Easy to use by novice faculty
Flexible for use by expert designers
Student and Admin views
Wizard-based admin
80% Solution for first pass
Good for supporting most courses
 Lecture and seminar best supported
Component Tools
Public/Private Homepage
Public/Private Syllabus
Course Materials
Assignments (Quiz, Problem Sets, etc.)
Grade Book
History and Near Future
’98-Research Project for HumBio Program
’00-Extended for Language Quizzing
’01-Full CMS Developed
‘02 Winter-Deployed for all Stanford faculty
‘02 Fall-Version 2 with links to Registrar
‘03-14,000 users; 1200 Courses Total
‘03-Evaluation by other institutions
‘03 Summer-Open Source Release
Ratings of Experience with CourseWork by Discipline
very good
very poor
Usefulness of CourseWork Tools by Discipline
Usefulness of t ools by discip
very useful
U sef u ln ess ( 3=very, 0=n o t )
not useful
Mater ials
Tech Info
CourseWork System Requirements
Web Server (Apache)
Java Servlet Application Server (Tomcat 3)
SQL-conformant database (Oracle 8)
DTL (a free HTML template application)
…running on Sun Solaris
Other Tested Environments:
Tomcat 4
CourseWork Demo
CourseWork Questions . . .
Assignment & Assessment Tool
Assignment & Assessment ToolAAM
Gives online tests, quizzes, problem sets,
formative assessments
Extensible and modular
Utilizes OKI Common Service APIs & Class API
Informs development of Assessment API
Utilizes IMS Question and Test Interoperability
(QTI) metadata specification.
Pedagogical Features
Supports discipline and teaching
method-specific extensions
Supports formative assessment (results
used immediately to help learning)
“Chunkable” assessments-questions not
locked into independent tests
Method for giving maximum flexibility with
maximum usability
Created by instructional technologists and
advanced users – no programming involved
Hide complexity from instructors and students
Give flexibility to end user in features
Templates - Flexibility in the
layout of an assessment
All questions, responses, and feedback can be in
multiple media
Questions can be accessed randomly or restricted to a
fixed sequence with no returns to earlier questions
Instructor can allow students to bookmark questions
and return to them
Web pages can be configured to contain a single
question, a set of questions, or the full assessment
Number of submissions by each student can be fixed
or unlimited
Templates - Flexibility in
evaluation views and procedures
Instructor can select type of evaluation: comments (at
part, question, or assessment level) or scores
Scoring can be done with numeric, alphabetical,
check/minus values
Evaluation can be done with students remaining
anonymous (for surveys or blind scoring)
Evaluation can be limited by role (e.g., section scorer,
peer review)
Multiple options for handling late submissions
(allowed-tagged, disallowed, allowed-no penalty)
Templates – Flexible access to
assessments, scores, and feedback
Release of assessment to learner can be immediate,
time based, or contingent upon completion of another
Retraction of assessment can be by date or upon
completion of assessment
Release of feedback can be immediate or by date
Release date, retraction date, feedback release date,
score release date, and due date can be set
Assessment can be released to specific groups in
class (e.g., sections)
Assessment or question-taking duration can be set to
specific number of minutes
Discipline-specific Extension:
Language Instruction
Multimedia questions
Recorded oral responses
Virtual conversations
Oral proficiency testing
Formative assessment
Easy, quick preparation of tests with
oral questions by instructors
Grading flexibility,ease, speed
Teaching-method SpecificSupport
for Large Lecture Science Courses
Support for sections and TA workflow
Annotated question pools for continuity as
TA’s change
Grading and Reporting flexibility
Integrated e-mail alert system
Special Question Type that has rationale with
multiple choice
M-Choice auto-scored; Rationales sorted for easy
review; Most frequently missed questions analysis.
Type your rationale for your answer here.
“Chunked” Assessments
Questions and response input can be included
in other activities
Stand-alone tests are non-mandatory
Central processing and reporting of results
Asynch Class Assessment Techniques in
schedule, email, announcement (probe,
muddiest part, etc.)
Fuzzy line between tutorial authoring and
materials creation when assessments can be
Interactions with other Tools
Display or get a question requested by
another tool (poll in discussion, question in
content, announcement, etc.)
Pass a score to other tool
Send feedback message to student(s) in
email, announcement, FAQ, IM
Post test date on schedule, syllabus
Link to a discussion
Final Questions . . .
More info:
Send e-mail to
Web Page

Open Source Learning Tools