Capstone: Vision, Video and VR
CSC 59866CD
Fall 2004
Lecture 3
Project Topic 2:
Virtualized Classroom
Zhigang Zhu, NAC 8/203A
http://www-cs.engr.ccny.cuny.edu/~zhu/
Capstone2004/Capstone_Sequence2004.html
Virtualized Classroom
Automated Production
Media Integration and
User-Customized Presentation
Zhigang Zhu1,2, Chad McKittrick1 and Weihong Li2
Video Computing Laboratory
Department of Computer Science
1City College and 2Graduate Center
The City University of New York
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
3
hypothesis, strategy & motivation
new pedagogy +
new technology
= improved
educational outcomes
1.traditional distance learning
2.support for students and faculty in an interdisciplinary center
3.integration of on-campus and at-a-distance students using
an embedded “intelligent” design environment
4.active, peer and cooperative learning in introductory courses
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
4
In cooperation with
Ellen Smiley
Dept of Psychology, School of Social Science
City College of New York
W. Richards Adrion
James F. Kurose
Edward M. Riseman
RIPPLES Group and Vision Lab
University of Massachusetts Amherst
Dr. Parviz Kermani
e-Learning Lecture Navigator (ELN)
Polytechnic University of New York
and IBM T.J. Watson Research Center
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
5
UMass Multimedia Asynchronous Networked Interactive Courseware
CD-MANIC: the best of both worlds
Supports a Rich Variety of Media

CDs


internet for:






for bandwidth intensive
content
content updates
logging student behavior
collaboration
search
external links
overcomes any bandwidth
constraints & provides
convenience


10/3/2015
proven technology & wide usage
Video Instructional Program
National Technological University
PINY, NCState, UMn, CCNY
contents don’t change often
use anywhere
 planes, trains, behind
firewalls
MDDE2004 @Zhigang Zhu /CUNY
6
CUNY and CCNY

CUNY – 20+ colleges


Ph.D. programs in GSUC
CCNY – 20, 000 students


MS programs only
Minority Serving Institute




In 2002, URM 66% in engineering, 35% in CS
US News 2003: No 2 in diversity
large percentage of work students
Questions:

10/3/2015
Effective learning tools for recruiting and retention
MDDE2004 @Zhigang Zhu /CUNY
7
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic MM capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
8
Multimedia Virtual Classroom

Automatic data collection and indexing




Audio/video
PPT, handwriting
online materials, etc
Collaborative Learning Environment (CLE)




10/3/2015
Active
Coversational
Constructive & reflective
Intentional & contextual
MDDE2004 @Zhigang Zhu /CUNY
9
MVC as an active CLE

active
Goal: active
interface,
navigation
base: index,
simple search
+ extended
search,
active objects
(text, images)
+ learnercontrolled
interface
• search
• relevance ranked
• 2nd level to repository,
Web
• audio, images
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
Scene
Selection
10
MVC as an conversational CLE
conversational
goal: support collaboration
base: threaded discussions
+ CLIMANIC
10/3/2015
chat
MDDE2004 @Zhigang Zhu /CUNY
shared
controls
11
MVC as a constructive & reflective CLE
constructive & reflective
goal: notation, collaboration
base: none
+ notes on slides, threaded discussions,
save CLIMANIC collaborations, integrated
reporting
HIST
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
12
MVC as an intentional & contextual CLE
intentional & complex & contextual
goal: problem-focused pedagogy & active classroom
strategy, realistic problem domains
base: problem-focused learning
+ CASA, on-campus & distance learning integration,
design tools & environment integration
Integrated
Tools
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
13
MVC supporting learner logger
important tool for
assessment
goal: assessing individual
and course learning goals
base: quizzer, logger
+ integrate with new
tools: extended search,
notation, collaboration
+ extend to integrated
design tools &
environment
+ integrate with CMS
(OWL, WebCT,
Blackboard)
report
generator

10/3/2015
Qui ckTime™ and a
TIF F (Uncompressed) decompressor
are needed to see t his pict ure.
Qui ckTime™ and a
TIF F (Unc ompress ed) dec ompress or
are needed to see t his pict ure.
logger server
internet
Qui ckTime™ and a
TIF F (Unc ompress ed) dec ompress or
are needed to see t his pict ure.
MDDE2004 @Zhigang Zhu /CUNY
Qui ckTime™ and a
TIF F (Unc ompress ed) dec ompress or
are needed to see t his pict ure.
Qui ckTime™ and a
TIF F (Unc ompress ed) dec ompress or
are needed to see t his pict ure.
14
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic MM capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
15
CCNY Virtualized Classroom

Automated MM data capture


Intelligent MM data integration


Sensing, synchronization and recording
Video, PPT slides, Mimio handwriting pages
Flexible User Interface

Course adaptive, user customized
Virtualized Classroom (VC)


10/3/2015
A new e-learning environment generated and
enhanced from materials captured from real
classrooms.
MDDE2004 @Zhigang Zhu /CUNY
16
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic MM capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
17
conversion
VC: Automatic MM capture
index
PPT Capture Add-on
Quic kTim e™ and a
TIFF ( Uncompressed) decom pressor
are needed to see this picture.
registration
Mimio Virtual Ink
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
video transformation
Qu ickT im e™ an d a
T IF F (Un co mp re ss ed ) d ec om p re ss or
a re ne ed ed to s ee th is pic tu re .
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Remote Reality
Omnidirectional Camera
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
18
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic MM capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
19
VC: Media Integration

PPT@ Slide & Mimio page registration



Panoramic Video Mosaicing


Pre-calibration
Dynamic calibration
From a panning camera
PPT@ Slide & video image registration

10/3/2015
Toward a 3D virtual environment
MDDE2004 @Zhigang Zhu /CUNY
20
VC: Media Integration (1)
PPT@ Slide & Mimio page registration
A simple approach: mark four corners …
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
21
VC: Media Integration (1)
PPT@ Slide & Mimio page registration
A simple approach: mark four corners …
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
22
VC: Media Integration (1)
PPT@ Slide & Mimio page registration
Question: What if one of the devices move?
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
23
VC: Media Integration (1)

Dynamic registration of PPT@ Slide &
Mimio page because

Hard to mark in some cases



Instructor may forget to do it
Devices may not be able to keep stationary


due to occlusion and height
Camera, projector, Mimio capture bar
completely automatic and “mind-free”
approach
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
24
Handwriting pages
H
video
V1
PPT slides
S
V2
combined
VC: Media Integration (2)
Panoramic mosaic from video
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
26
VC: Media Integration (2)
Panoramic mosaic from video
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
27
VC: Media Integration
Toward an immersive VC interface
Active WWW link
Synthetic Hi-Res digital display
Slide combined with video
Panoramic mosaic from video
Annotated
and active
instructor
Question Window, ToC, etc
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
28
VC: Media Integration (3)
Replacing the slide projections
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
29
VC: Media Integration (3)
Image Alignment
From Video
From Digital Slide (PPT)
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
30
VC: Media Integration (3)
Image Alignment
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
31
VC: Media Integration (3)
Instructor Extraction…
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
32
VC: Media Integration (3)
Instructor Extraction…raw
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
33
VC: Media Integration (3)
Instructor Extraction…refined
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
34
VC: Media Integration (3)
Instructor Extraction…contour
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
35
VC: Media Integration (3)
Instructor Extraction…and integration
video
shadow
10/3/2015
image
contour
MDDE2004 @Zhigang Zhu /CUNY
36
VC: Media Integration
Toward an immersive VC interface
Active WWW link
Synthetic Hi-Res digital display
Slide combined with video
Panoramic mosaic from video
Annotated
and active
instructor
Question Window, ToC, etc
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
37
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic MM capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
38
User Customized Presentation

Virtualized Classroom Presentation System
(VCPS) in Java, consists of



An authoring tool – VCPS creator
An presentation interface - VCPS player
Features



10/3/2015
Customized sizes and locations
Pop up ToC for indexing
Synchronized play, next, last…
MDDE2004 @Zhigang Zhu /CUNY
39
User Customized Presentation

PPT+ panoramic view + popup ToC
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
40
User Customized Presentation

PPT & Mimio integrated +Popup ToC
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
41
User Customized Presentation

PPT+ Mimio + Combined + images +New Buttons
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
42
User Customized Presentation

PPT+ Mimio + Combined + images +New Buttons
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
43
User Customized Presentation

PPT+ Mimio + Combined + images +New Buttons
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
44
User Customized Presentation

PPT+ Mimio + Combined + images +New Buttons
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
45
User Customized Presentation

PPT+ Mimio + Combined + images +New Buttons
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
46
User Customized Presentation

PPT+ Mimio + Combined + images +New Buttons
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
47
Outline


Background
A Bigger Picture


CCNY Virtualized Classroom




Multimedia Virtual Classroom (UMass+CCNY)
Automatic MM capture
intelligent media integration
user-customized presentation
Conclusions and Discussions
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
48
Conclusions and Discussions

CCNY Virtualized Classroom




automatic MM capture
intelligent media integration
user-customized presentation
Multimedia Virtual Classroom (UMass+CCNY)



Active and collaborative learning
Content-based searching and indexing
Pedagogy and assessment
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
49
Demos


CCNY Virtualized Classroom System
UMass CD-MANIC with enhanced
searching
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
50
Capstone Project

Java and Java 3D

Sun ONE Studio 4 CE
Java 3D has a high level implementation of OpenGL
http://java.sun.com/products/java-media/3D/


Technical Aspects – Do and Don’t






Interface: 2D or 3D
Video, and Omnidirectional Video
Indexable, Searchable pages for active learning
Learner log for evaluation
Internet enabled vs Standalone
Reporting and Publications

10/3/2015
Conference and Journal Papers
MDDE2004 @Zhigang Zhu /CUNY
51
References



Z. Zhu, C. McKittrick and W. Li, Virtualized Classroom – automated
production, media integration and user-customized presentation, The 4th Int.
Workshop on Multimedia Data and Document Engineering (with CVPR’04), July
2, 2004, Washington DC
W. Li, H. Tang and Z. Zhu, Automated registration of high-resolution images
from slide presentations and whiteboard handwritings via a low-cost digital
video camera. The 2nd IEEE Workshop on Image and Video Registration (with
CVPR’04), July 2, 2004, Washington DC
W. R. Adrion, J. Kurose, E. Riseman, Z. Zhu, The Multimedia Virtual
Classroom – Developing and Evaluating A Constructivist Learning
Environment, Fifth International Conference on Information Communication
Technologies in Education, July 1-3, 2004, on Samos Island, Greece
Links of the first two are on Prof. Zhu’s Publication List
10/3/2015
MDDE2004 @Zhigang Zhu /CUNY
52
Next

After Class


Update the system design
Next Lecture

10/3/2015
Visual Sensors
MDDE2004 @Zhigang Zhu /CUNY
53
Descargar

3D Virtualized Classroom - City University of New York