The "Virtual TA" Android App (P5a)

The application you will be constructing is a math tutoring application that is powered by a virtual teaching assistant (vTA). The expected application usage is as follows. Initially, when the application is opened the user is greeted with a welcome screen at which point they can elect to start a tutoring session or terminate the application. A tutoring session is comprised of 10 different problems where the first 6 problems are selected by the vTA and the remaining 4 by a testing algorithm. The goal of the vTA is to select the 6 best training problems (i.e. a customized lesson plan) in order to prepare the user for the test. It is your job to construct the vTA.

The vTA at its core is a service that contains a reasoning engine. The vTA is responsible for making the following three decisions...

  1. What question should be presented the user in order to best prepare them for the exam?
  2. Is the user shown how difficult the problem was after they answer the question?
  3. What type of feedback should be requested at the end of each question to help the vTA make better decisions?

As the developer of the vTA you are responsible for designing the heuristic that makes those decisions. In order to make those decisions you will need to gather data to make an informed decision. Your vTA must consider the following information when making a decision...

Grading Criteria

This assignment is part of a research study being conducted by the Service Oriented Computing Lab at NCSU. Therefore, an integral part of the assignment is the completion of pre and post project surveys as well as keeping a record of your progress throughout the assignment. Part 3 of the project description is dedicated to describing how these surveys are to be filled out and when and where they are to be submitted.

Grading Rubric
Pre-Project Survey 5%
Post-Project Survey 10%
Project Worklog 35%
Heuristic: Exam 20%
Heuristic: Environment 10%
Heuristic: Emotion 20%

Part 1: Setup Your Android Development Environment

You should all be familiar with the basics of an android application from your previous assignment. To enable you to focus on the vTA's design and implementation we have provided the core components of the application in an existing eclipse project. Now, to import the Android project into Eclipse use the following instructions....

  1. Download vta_app.zip
  2. In Eclipse go to File -> Import...
  3. Select General -> "Existing Projects into Workspace" and click Next
  4. Select "Archive File" and browse to the directory where you downloaded vta_app.zip
  5. Click Finish and the project will import into your workspace
  6. Rename the project to: [unity_id]_Virtual_TA

The entire application will function within a device emulator that is packaged with the android SDK. Additionally, you will be issued an android device which you will be able to run your program on. These devices are running Android 1.6 (Donut), so your application must be built using the 1.6 SDK, which you can download in eclipse using Window -> "Android SDK and AVD Manager".

Main Application Components

User Interface

As you probably noticed in the grading rubric there are no points awarded for the UI. Almost all the UI components you need are already provided. The only thing that you need to do is provide a means for user's to give feedback to the vTA at the end of a question. This can be done using something as simple as a set of radio buttons. When you do add UI components you must add them to the /res/layouts/vta_answer.xml file. You can then configure the components and gather the results using the appropriate methods in VirtualTA.java.

Android Hints

Part 2: Developing the vTA

The vTA is a component of the application's ExamService. Though we stated it in the introduction it is worth saying again, the goal of the vTA is to select the best training problems in order to prepare the user for the test. In essense you are developing a customized lesson plan for your user that gives them the best chance of sucess. Your vTA could lookup the user's standardized test scores (e.g., SAT or GRE), compare their age and gender to the national math scores of those in their demographic, or even look to see if any members of their family are world renown mathematicians. The good news for you is that we aren't asking you to go that far. We would however like you to consider 3 categories of information as you design the logic of your vTA. Each category is described in its own section below and is followed by an example heuristic that shows how to combine all 3 in your vTA.

Before describing the categories we must first describe the defining characteristics of the math questions used. The questions used in our program are a subset of the question bank that is maintained by the National Center for Educational Statistics. The questions are extracted data gathered by the National Assessment of Educational Progress. In particular, we focus on multiple choice mathematics questions asked to 8th and 12th grade students. To help you distinguish between questions, each question has the following data associated with it.

Category 1: Exam Data

Looking at the user's past performance is a good indicator of what types of problems they do well on and where they struggle. Knowing the strengths and weakness of your user allows the vTA to tailor questions to their specific areas of need. The ExamService can be used to gather and inspect the contents of the current exam using getActiveExam() and older exams by using getRecentExams(...). If you feel that you need more information, you are free to add additional methods to the bottom of ExamService.java.

Category 2: Environmental Data

This application is operating on a mobile phone, which has a variety of sensors that give you insights into the user's current environment. For instance, our user might not perform certain tasks well in noisy environments, which we can detect using the microphone on the device. You don't need to utilize a sensor with that level of sophistication, but you must do something. It is recommended that you use the GPS sensor, since you all have had prior experience with it. You can use their location to make an assumption about their environment, which can be used by the vTA when making a decision.

For this part, the vTA is allowed to make assumptions about your user. For example, you could hard code coordinates that represent the location of a nearby bar and the local library. The vTA could then implement a heuristic that determines that when users are at a bar they are more likely to need easier questions due to intoxication, but can answer harder questions when at the library. Use your imagination and be creative, but make sure your decision has a reason that can be tied back to your overall goal!

Category 3: Emotional Data

The user's emotional state can be directly correlated to their ability to learn. In fact, Dr. Lester of our Computer Science department has many publications on the effect that emotion has on the user's ability to learn. For the purpose of this study, take the following theory to be true, "students learn better in situations where they are neither extremely bored or frustrated, but instead at a median between the two". A student is assumed to be bored when the problems are too easy and anxious or frustrated when the problems are too hard. The theory is credited to Csikszentmihalyi who named the sweet spot between boredom and anxiety, the flow channel. You can get an overview of his theory here.

How do we tell how the user is feeling? You can ask them. This is the only time you should add UI to the system (see User Interface). You can take the feedback from the UI to inform the vTA about the user's current emotional status. That information must then be used in two ways.

  1. You must use their emotional state to help select the question they are to be asked next.
  2. You must also store the data in order to make future predictions about how a user will respond to a specific question type.

Each team will also be given individualized instructions that will further guide them on completing this task. These instructions will be given shortly after the pre-project surveys are completed (see Milestones and Deliverables).

Example Heuristic

An example heuristic for the getNextQuestion() of the vTA would be as follows (Do NOT use this exact heuristic!).

  1. Start with equal probability of asking question from any grade level, difficulty, or content area.
  2. If past exams was better at one grade level then increase probability of asking question from the other grade level
  3. Same as #2 but with difficulty
  4. Same as #2 but with content area
  5. Increase difficulty if user is in a quiet environment
  6. If user is frustrated decrease difficulty and/or grade level
  7. If user is bored increase difficulty and/or grade level
  8. Select question using the probabilities
  9. Inspect question to see if it has been asked before and if so repeat step 8
  10. If question will frustrate or bore user too much repeat step 8 after adjusting probabilities
  11. return the question

Part 3: Recording Your Progress

50% of your grade is a result of recording your progress in addition to filling out the pre and post project surveys. Both the surveys and the worklogs will be graded based on two criteria, which are completeness and timeliness. There are no right or wrong answers and the contents of the surveys will not in any way effect the grade you receive for the quality or completeness of your code. Further, all surveys and worklogs are to be completed individually and should contain only data relevant to you, not your partner!

The pre-project survey is to be completed prior to implementing any code for the project. Similarly, the post-project survey is to be completed after finishing the project. Finally, the worklog is a short form that is to be filled out every time you work on the project. The worklog is short so it should only take about a minute to fill out, but for the purposes of this study it is the most important piece of information. You can condense an entire days progress into one worklog, but it must be submitted when you are done working for the day. You can submit more than daily if it helps keep you organized.

Milestones & Deliverables

04/14/2010 -- Last day to submit your pre-project survey and pick up a device.

04/15/2010 -- You will be given individualized instructions for how to gather the emotional data.

04/23/2010 -- Last day to submit any worklogs from the previous week.

04/30/2010 -- Last day to submit any worklogs from the previous week.

04/05/2010 -- Submit the following via wolfware...

Questions??

If you have any questions about the instructions above or anything else related to this assignment please contact Derek Sollenberger. Derek will be available to answer you questions via email djsollen_at_ncsu.edu and can also schedule appointments anytime Wed-Fri to meet and discuss any issues you may be facing.