COMP 410 Update But Connecting is Only One Piece… • How do you interpret massive amounts of data? • Where do you store this data? • How do you transmit this data? • Where do you analyze this data so you can understand it? • What happens when a device disconnects intentionally or unintentionally? • How does another device access and use this data? • How can we make our solution extensible? The Core Issues High volume of data It’s infeasible to have every sensor send every reading to every decision agent Solution: Intermediate nodes to aggregate, analyze, and summarize sensor information Disconnections Our theoretical real-time always-connected data pipeline isn’t realistic If devices disconnect, intentionally or unintentionally, information can be lost Solution: Intermediate nodes store the information they receive, so they can re-emit it later Different kinds of devices Third-party sensors and actuators often speak different languages Solution: An extensible framework that can use any communication protocol High-Level Framework Design Framework Application Layer, a.k.a. Business Logic Network Protocol Storage Protocol Disconnection Protocol Solving: High Volume of Data Framework Application Layer, a.k.a. Business Logic Network Protocol An intermediate node receives sensor data, analyzes it in the business logic, and sends a summary to decision agents. Storage Protocol If historical data is too big to store locally, store it in the cloud. Disconnection Protocol Solving: Disconnections Framework Application Layer, a.k.a. Business Logic Network Protocol Storage Protocol Disconnection Protocol We can use historical data to automatically send messages once a device (re)connects. On disconnection, we can notify a monitoring service, and attempt to reconnect if necessary. Solving: Different Kinds of Devices Framework Application Layer, a.k.a. Business Logic Network Protocol We’ll provide standard internet connections, but anything can plug in here. Storage Protocol Disconnection Protocol Timeline Prototype 0 3/13 Prototype 0 Prototype 0 1) Can connect devices with different protocols ◦ Fundamental to the project so that data can eventually be transferred 2) Can transfer and interpret data sent between devices ◦ Can handle intermittent, predictable disconnection ◦ Can store data in local storage and using cloud services Prototype 0 Technologies • Wunder Bar • Demonstrates real time data access through a cloud service • Demonstrates unreliable connections • Kinect • Demonstrates direct, reliable connection • Intermittent data sending • Laptops (2x) • Demonstrate wireless connection • Holds business logic • Demonstrates local storage being pushed to a database intermittently Prototype 1 3/27 Prototype 1 Prototype 1 1) Heavy focus on data storage, analysis, and interaction with cloud services ◦ Now that devices are connected, we want to make sure we can use the data effectively 2) Added storage options more applicable to big data problems ◦ Demonstrating the extensibility of the Framework Prototype 1 Technologies Windows Azure ◦ SQL database for storage ◦ Virtual machines for computation ◦ Machine learning for analysis Hadoop ◦ Demonstrates large scale data aggregation Prototype 2 4/10 Prototype 2 1) UX and UI added related to the Hurricane Story ◦ Now that data can be sent and stored, implementing the business logic to solve a real problem ◦ Human interaction with the data to make decisions Prototype 2 Technologies Web Applications ◦ Provide interfaces for users to interact with data ◦ Display data in human readable format Mobile Devices ◦ Add new networking technology to the system ◦ Demonstrates extensibility of the design Final Project 4/24 Final Prototype Full integration of all technologies ◦ All protocols are in place so that devices can communicate through the Framework Complete solution for the hurricane problem ◦ UX/UI refined for human interaction Increased Scale from Prototype 2 ◦ Demonstrates ability of our Framework to tackle large scale data problems Final Prototype Technologies External Weather Database ◦ For pulling big data for simulation ◦ Demonstrates full power of data handling technologies Questions?