Some research ideas on speaking skill improvement through video reflection

I'm kind of in groping mode for a new major research project, possibly leading to a dissertation, so here goes some brainstorming. So far I've produced scattered work in the fields of materials development, spoken fluency, self-assessment of speaking by learners, feedback on essay writing, self-confidence in language learning, and teacher training--and it is time to launch into a fairly involved study that will define my main specialty. So what will that be?

My main general interest is how to help people develop intercultural communication competency, with a focus on methods for developing skills in spoken fluency and confidence. At my university, I'm part of a team coordinating curriculum development for academic speaking of English for Japanese college students, and one of the most exciting developments is the video-recording based self-assessment system we will pilot from April.

What kind of research could we do there? Let me type as I think here...

From April to June, I'll be helping 40 some new college freshmen learn basics for "English academic speaking" ie the oral communication skills they will need to survive in English language medium college courses at ICU. The syllabus will start with skills/phrases for basic greetings, self-intros, small talk and then move on to skills for participating in or leading class discussions (lots of sub-skills there). It will touch on presenting, but not in a big way aside from presenting an organized opinion as part of a discussion.

One part of the curriculum will be a "self-assessment" that students will do based on video-recording, watching, and analyzing their own speaking to set goals and improvement strategies. What this basically means is that we'll have groups of students come into a recording room, take turns leading or participating in small group discussions, video-record them doing it, give them the video file to watch, and then ask them to notice what areas they especially have difficulty in...and they will probably come up with a bunch of typical things such as "I lack fluency" or "My voice is hard to understand" or "My opinion didn't really get across to the other person" "I have broken grammar--We'll have a fairly long checklist for this kind of thing and fairly difficult? task to "flush out their bugs". After they self-identify their areas of difficulty (and we are assuming these self-analyses will be basically valid--accurate enough?--...they won't be saying "My main problem is only grammar" when actually their main problem is pronunciation. In any case, the most important thing is that each student finds some clear goals for working on their speaking skills over the next year or so in the intensive English program. Also, based on the goals they set such as "I want to improve my pronunciation (my intonation, R-L difficulties etc.)," they need to find specific and realistic practice plans/methods they can try to implement independently. "I will listen to podcasts from xxx on every Thursday" and then actually do those plans, and actually make improvement.

Here's the critical issue: If the reflective process of Record->Watch->Self-analyze->Set goals/plans->Implement-> Review/Reflect on improvement (Measure progress in focus areas)-> Modify or Continue implementation does not really help learners improve their speaking, or if it is impossible to really measure any improvement based on the process, then arguably it is a waste of effort. Last year's pilot and survey results showed, at least, that learners feel they benefit from watching and analyzing their own speaking. This year's project, ideally, needs to go one more step and demonstrate that valuable progress is being made in the areas of focus. Of course, the inherent complexity of measuring language acquisition progress has to be considered. To be realistic, a simplified pre/post measurement of a isolated task/skill area, even with control groups etc. proves very little about whether the learner has become a better speaker.

So...what are some good research questions here:

1. In what way do foreign language learners benefit from self-analysis of their own speaking performance? Clearer goals? More self-awareness/monitoring when speaking? What do the learners say (based on questionnaires/follow-up interviews)
2. Is video-watching / reflection really needed? How much? One time? I've only seen myself speaking Chinese once and I hated it (it was ugly) Maybe I should eat my own dog food more. Is it a waste of time and effort? Are we going down the wrong alley here? Isn't it better to just practice, practice, practice?
3. Can learners identify their weaknesses? (like a teacher would) Do they need to?
   
4. Can learners set effective practice plans for themselves? (given resources)
5. Will learners at my institution actually do the self-study that they set out to do? (Or will they just be listing a bunch of bull to appear to engage in a reflective process)
6. Why do we do it two times in the term? Are they supposed to see improvement? Do they? (Sure, they get used to the tasks (organizing an opinion with appropriate signal phrases, leading a discussion etc., so their performance obviously improves, which is a good thing.) Does doing it two times lead to encouragement to continue the cycle of reflective learning/improvement?
7. Can engagement in the learning process be the main basis for the grade in the class? (I wish we didn't have to give grades in the course, but we have to, so...) How will that work? How do we use the video and self-assessment to determine whether a student has worked hard/made progress in the areas we want them to? Can we use a portfolio system? (record, self-evaluate, reflect, set goals, reflect on practice methods) Is that too much writing in a speaking class? Can the reflection etc. be in speaking, like a speaking journal with an online recorder function? Hmm...maybe in the future.
   

A little tangent here. What should we be calling this process of Record-Watch-Analyze-Set Goals/Plans-Do it-Reflect-Modify? Last year we misnamed it the Academic Speaking Assessment, and we had many students and teachers completely misunderstanding that we were giving them a test with a score about how good their speaking is. We should at least call it the Self-Assessment, but even that is just one part. It is more like a Academic Speaking Improvement Cycle/Process which is evaluated by a portfolio of measurement, analysis, goal-setting, practice and reflection. The assessment (for the grade of the class) is just whether they engage deeply and thoughtfully in the process of improving their speaking skills.

Maybe we shouldn't call it anything and just explain. "The best way to improve your speaking is not only to just practice, but have a cycle of... In this class, your grade will be determined largely by how well you complete two cycles of reflective learning about your own speaking..." Something like that?