Science Learning Environments : Assessment, Effects and Determinants - Barry J. Fraser
Why Use Short Forms Of Instruments - less scales and less items (about 25)
(a) less time to administer and score
(b) easy hand scoring
(c) adequately reliable for applications that require the class means
Perceptions of the whole class (average result) can be different from the individual's perception.
Stages In Improving Classroom Environments
(a) Assessment - Pupils' reponse to instrument
(b) Feedback to teacher on pupils' responses
(c) Reflection and Discussion by teachers
(d) Intevention
(e) Reassessment - to see if interventions brought any improvements
Additional Uses Of Instruments (add on to list in previous reading)
(a) Teacher education
(b) Teacher assessment
Thursday, December 6, 2007
Learning Environments Research : Yesterday, Today and Tomorrow
Learning Environments Research : Yesterday, Today and Tomorrow - Barry J. Fraser
Uses Of Classroom Environmental Instruments
(a) Association between students' cognitive and affective learning outcomes and their
perceptions of psychosocial characteristics of their classrooms.
(b) Source of process criteria in the evaluation of educational innovations (compare innovative
classes with control group using the items for psychosocial perceptions of their learning
environments
(c) Differences between student and teacher perceptions in the same actual classroom envt
(d) Differences between student preferred and actual perceptions of the classroom envt
(e) Scales in instruments used as criterion variables in research aimed at identifying how the
classroom envt varies with factors such as teacher personality, class size, grade level,
subject matter, nature of the school level envt and type of school.
(f) Qualitative instruments (observations, interviews, student diaries, open-ended questions in
questionnaires, etc) can replicate findings of quantititive questionnaires and clarifying replies
to the questionnaires
matter, nature of the school level envt and type of school.
(g) Enable cross-national studies which can highlight differences between practices in different
countries which can be questioned or used as a learning point for improvement
(h) Feedback from perception studies can be used to guide improvements in the classroom
environments
Uses Of Classroom Environmental Instruments
(a) Association between students' cognitive and affective learning outcomes and their
perceptions of psychosocial characteristics of their classrooms.
(b) Source of process criteria in the evaluation of educational innovations (compare innovative
classes with control group using the items for psychosocial perceptions of their learning
environments
(c) Differences between student and teacher perceptions in the same actual classroom envt
(d) Differences between student preferred and actual perceptions of the classroom envt
(e) Scales in instruments used as criterion variables in research aimed at identifying how the
classroom envt varies with factors such as teacher personality, class size, grade level,
subject matter, nature of the school level envt and type of school.
(f) Qualitative instruments (observations, interviews, student diaries, open-ended questions in
questionnaires, etc) can replicate findings of quantititive questionnaires and clarifying replies
to the questionnaires
matter, nature of the school level envt and type of school.
(g) Enable cross-national studies which can highlight differences between practices in different
countries which can be questioned or used as a learning point for improvement
(h) Feedback from perception studies can be used to guide improvements in the classroom
environments
New Learning Environments And Constructivism - The Student' Perspective
New learning envts and constructivism - The student's perspective - David Gijbel, et al
I did not do a review of the article but was more interested about the different types of learning envts and so picked up some points on that and recorded it here.
New Learning Envts
(a) Project-based education
(b) Case-based learning
(c) Problem-based learning
(d) Task oriented learning
(e) Practical work
(f) Work place laerning
(g) On-line learning
Features Of Problem-Based Learning
(a) Student centred
(b) Small student group under guidance of facilitator
(c) Tutor as facilitator
(d) Authentic problem
(e) Problems encountered used as a tool to achieve required knwledge and the problem solving
skills necessary to eventually solve the problem
(f) Self directed learning
Features That Can Enable The Constructive Building Of Knowledge
(a) Arguments, discussion, debates
(b) Conceptual conflicts and dilemmas
(c) Sharing ideas with others
(d) Material and resources targetted towards solutions
(e) Motivation towards reflections and concept investigations
(f) Meeting students' needs
(g) Making meaning, real-life examples
Reflection
PBL is a great tool but the problem is constructing an authentic problem. Many of us are not creative or knowledgable enough to create the problem which has all the dimension mentioned above. If MOE could build up resources that we could implement, more teachers would embark on these new learning envts.
If we used all the new learning envts mentioned above, then I will say we are a true educator and not just a person who dessiminates information. Learning would be so exciting. However, the type of learning you use must be suitable to the type of pupils you are teaching. Some of these methods are quite high leve and pupils would need a lot of scaffolding and hand-holding. But imagine the bebfits if we can train pupils to think for themselves.
However, even if we have the lesson plans or case studies, etc provided to us, that does not mean we would know how to implement them in an effective way that allows for maximum learning. These forms of learning require skills that teachers need to develop otherwise the lessons would just be a waste of time and degenerate into a chaotic mess.
I did not do a review of the article but was more interested about the different types of learning envts and so picked up some points on that and recorded it here.
New Learning Envts
(a) Project-based education
(b) Case-based learning
(c) Problem-based learning
(d) Task oriented learning
(e) Practical work
(f) Work place laerning
(g) On-line learning
Features Of Problem-Based Learning
(a) Student centred
(b) Small student group under guidance of facilitator
(c) Tutor as facilitator
(d) Authentic problem
(e) Problems encountered used as a tool to achieve required knwledge and the problem solving
skills necessary to eventually solve the problem
(f) Self directed learning
Features That Can Enable The Constructive Building Of Knowledge
(a) Arguments, discussion, debates
(b) Conceptual conflicts and dilemmas
(c) Sharing ideas with others
(d) Material and resources targetted towards solutions
(e) Motivation towards reflections and concept investigations
(f) Meeting students' needs
(g) Making meaning, real-life examples
Reflection
PBL is a great tool but the problem is constructing an authentic problem. Many of us are not creative or knowledgable enough to create the problem which has all the dimension mentioned above. If MOE could build up resources that we could implement, more teachers would embark on these new learning envts.
If we used all the new learning envts mentioned above, then I will say we are a true educator and not just a person who dessiminates information. Learning would be so exciting. However, the type of learning you use must be suitable to the type of pupils you are teaching. Some of these methods are quite high leve and pupils would need a lot of scaffolding and hand-holding. But imagine the bebfits if we can train pupils to think for themselves.
However, even if we have the lesson plans or case studies, etc provided to us, that does not mean we would know how to implement them in an effective way that allows for maximum learning. These forms of learning require skills that teachers need to develop otherwise the lessons would just be a waste of time and degenerate into a chaotic mess.
Wednesday, December 5, 2007
Classroom envt research : Progress & possibilities
Classroom environment research : Progress and possibilities - Jeffrey Dorman
Without consideraton of the crucial role of the psychosocial environment of classrooms, educational productivity cannot be optimised.
Historical perspective
Thomas - recorded explict classroom phenomena
Levin - huamn beh is a function of th perons and the environment
Murray - needs important determinants of beh within a person
- press of an object is what it can do to the subject - power to affect the well being of the subject
Pace - defined envt - overall atm, characteristics, kind of things that are rewarded, encouraged, emphasised, style of life which is valied by community and is most visibly expressed and felt
Stern -degree of person-envt congruence is related to student outcomes (the degree to which a pupil's needs is met will affect his beh)
Fraser - congruence between actual and preferred envts is assessed and related to student
outcomes
- development and validation of instruments to assess specific dimensions of the
classroom envts
Moos - three dimesions of human envt : R, P and S
Walberg - student perceptual data (collected by questionnaire) to determine if changes in teaching and learning affected the classroom climate - student judgements about classroom is a good indicator
Methodology
Quantitative - trained observers to code event (alpha press)
- student/teacher perceptions throught questonnaire (beta press)
Qualitative - ethnographic data collection methods
Previous Learning Envtal Research
(a) Development and validation of instruments - 9 genreal types
- several specific types for particular settings
(b) Effect on classroom envt of antecedent variables
(c) Use of envtal assessments in teacher education
Current Directions For Classroom Envt Research
(a) Multimedia learning envts
(b) Instrument development
(c) Introducing the study of learning envts in other countries
Future Directions For Classroom Envt Research
Without consideraton of the crucial role of the psychosocial environment of classrooms, educational productivity cannot be optimised.
Historical perspective
Thomas - recorded explict classroom phenomena
Levin - huamn beh is a function of th perons and the environment
Murray - needs important determinants of beh within a person
- press of an object is what it can do to the subject - power to affect the well being of the subject
Pace - defined envt - overall atm, characteristics, kind of things that are rewarded, encouraged, emphasised, style of life which is valied by community and is most visibly expressed and felt
Stern -degree of person-envt congruence is related to student outcomes (the degree to which a pupil's needs is met will affect his beh)
Fraser - congruence between actual and preferred envts is assessed and related to student
outcomes
- development and validation of instruments to assess specific dimensions of the
classroom envts
Moos - three dimesions of human envt : R, P and S
Walberg - student perceptual data (collected by questionnaire) to determine if changes in teaching and learning affected the classroom climate - student judgements about classroom is a good indicator
Methodology
Quantitative - trained observers to code event (alpha press)
- student/teacher perceptions throught questonnaire (beta press)
Qualitative - ethnographic data collection methods
Previous Learning Envtal Research
(a) Development and validation of instruments - 9 genreal types
- several specific types for particular settings
(b) Effect on classroom envt of antecedent variables
(c) Use of envtal assessments in teacher education
Current Directions For Classroom Envt Research
(a) Multimedia learning envts
(b) Instrument development
(c) Introducing the study of learning envts in other countries
Future Directions For Classroom Envt Research
(a) Provide more comprehensive evaluation of classrooms and school learning envts - include physical envt esp with the introduction of new technologies for a "smart nation" and "knowledge nation".
(b) Linking classroom envt with other learning envts (eg. home, out of school peer group, part time work) to get a more comprehensive understanding of student outcomes
(c) Embed classroom envt needs in large, comprehensive research projects
(d) More robust methods of data analysis
REFLECTIONS ON SESSION 5
Objectives for Session 5
(a) Evaluate the school learning environment through 3case studies
(b) How to use the software, SPSS, to analyse the data to determine if there is significant
difference between the preferred and actual perceptions for each scale in the questionnaire
(c) How to write a project report on the assessment of a learning environment
(d) Group assignment : To write a project report on the assessment of a learning environment
Reflections
We shared the group work carried out the previous day on assessing the school environment based on 3 case studies. There were some teachers who felt School A was a positive environment because of the strong discipline and emphasis on academic achievemnt. They felt that that were imprtant issues that needed to be put in place. I agree that discipline is an improtant issue. In fact I wrote on it in my second entry in Dr Quek's blog for session 1. Hoever, in this particular case study, I felt that the school had already established the discipline and academic results and that it was long overdue that they had stopped being one sidded in their education and had moved on to providing a holistic education and creating a nurturing environment. Both teachers and pupils were unhappy in the school and I felt that if the situation had continued on, before long, negative repercussions would start manifesting. Unfortunately, parents only see the good academic record and have no idea that the school environment was not one of learning and growing but of drilling for examinations only.
We then discussed the importance of Cronbach alpha coefficient and the paired sample T Test.
Cronbach alpha reliability
(1) To see if the items you use in the survey gives responses that are similar among all the
participants as well as in different situations.
(2) Why may the item not be reliable (have a low cronbach alpha reliability) :
(i) did not understand statement
(ii) interpretation of statement
(iii) may not be culturally suitable
Writing a report
Cover page
Abstract
Introduction
Lit review
Key words
Research questions - (i) learn more about the learning envt
- (ii) are the differences significant
Metodology - Sample size
- Instrument used
- Procedure
Findings - Tables - x3
Use only 3 lines in table
Table 1 : What the scales are about and what the items are
Table 2 : Cronbach's alpha reliability for each item (preferred and actual) for
each scale
Table 3 : Mean, Std dev and Item mean
- Line Graph -x1
(a) Item Mean against Items for both preferred and actual (2 line curves)
(b) Purpose of graph (i) study trend
(ii) see pre and post results for comparison
If there is no sig diff, add the 2 means and divide by 2 and get one point
If there is a sig diff, plot both means.
Analysis of results
Conclusion - main findings
- gaps to address
- recommendations
References -Book : Name of book in italics
- Journal : Name of article in journal in italics / Name of journal not in italics
Volume of journal in italics
Word, "page" or "pg" not used. Just write the number range
Report should be (a) 3-4 pages
(b) Spacing - 1.5
At first I wondered what was the point of writing a project report as report writing was our first module and this module was on classroom learning environment and not report writing. It was only later tonight, when I was writing the rationale for the group report that I realized that the point was not to teach us how to write a report but the report was the assessment product to detemine if
(a) we had read enough on the topic - Lit review
(b) we understood the meaning of learning environments and the importance of assessing the
learning environment so that we can create one that is the most conducive to pupils learning
as it meets the pupils' needs - Introduction, Rationale
(c) we know how to assess a learning environment
(i) know the methodology for assessing the learning environment - Sample size,
Instrument and Procedure
(ii) know how to use the software, SPSS, to find the Cronbach's apha coeeficient and the
paired sample T Test and understand their importance - Findings, Analysis
A great way to test our understanding instead of giving us the traditional test.
(a) Evaluate the school learning environment through 3case studies
(b) How to use the software, SPSS, to analyse the data to determine if there is significant
difference between the preferred and actual perceptions for each scale in the questionnaire
(c) How to write a project report on the assessment of a learning environment
(d) Group assignment : To write a project report on the assessment of a learning environment
Reflections
We shared the group work carried out the previous day on assessing the school environment based on 3 case studies. There were some teachers who felt School A was a positive environment because of the strong discipline and emphasis on academic achievemnt. They felt that that were imprtant issues that needed to be put in place. I agree that discipline is an improtant issue. In fact I wrote on it in my second entry in Dr Quek's blog for session 1. Hoever, in this particular case study, I felt that the school had already established the discipline and academic results and that it was long overdue that they had stopped being one sidded in their education and had moved on to providing a holistic education and creating a nurturing environment. Both teachers and pupils were unhappy in the school and I felt that if the situation had continued on, before long, negative repercussions would start manifesting. Unfortunately, parents only see the good academic record and have no idea that the school environment was not one of learning and growing but of drilling for examinations only.
We then discussed the importance of Cronbach alpha coefficient and the paired sample T Test.
Cronbach alpha reliability
(1) To see if the items you use in the survey gives responses that are similar among all the
participants as well as in different situations.
(2) Why may the item not be reliable (have a low cronbach alpha reliability) :
(i) did not understand statement
(ii) interpretation of statement
(iii) may not be culturally suitable
(3) Reliability above 0.5. However, 0.9 is not considered good as the item is too "ideal" and not realistic.
Paired Sample T Test
To determine if the difference between two samples (pre and post results) are significant or not. Significance level can be set at 95% or 97%. At 95% significance level, the hypothesis is that any result that gives a T test that is less than 0.05 means that the difference between the two samples is significant.
In terms of the assessment of the learning environment, T-test scores less than 0.05 for a 95% significance level means that the difference between the preferred and actual perceptions of the students is significant and so intervention measure should be planned and carried out to close this gap between preferred and actual so as to create a conducive learning environment.
Once the intervention measures are carried out, the survey should be done again to ensure that the intervention had been appropriate to address this area of concern.
The next step as to use the SPSS softare to calculate the paired sample T test for the survey we had carried out the previous day.
Dr Quek talked us through how to write a report using a sample report.
We then worked on our project. We used the data we collected and the analysis we had made over the 2 days and wrote a project report.
Writing a report
Cover page
Abstract
Introduction
Lit review
Key words
Research questions - (i) learn more about the learning envt
- (ii) are the differences significant
Metodology - Sample size
- Instrument used
- Procedure
Findings - Tables - x3
Use only 3 lines in table
Table 1 : What the scales are about and what the items are
Table 2 : Cronbach's alpha reliability for each item (preferred and actual) for
each scale
Table 3 : Mean, Std dev and Item mean
- Line Graph -x1
(a) Item Mean against Items for both preferred and actual (2 line curves)
(b) Purpose of graph (i) study trend
(ii) see pre and post results for comparison
If there is no sig diff, add the 2 means and divide by 2 and get one point
If there is a sig diff, plot both means.
Analysis of results
Conclusion - main findings
- gaps to address
- recommendations
References -Book : Name of book in italics
- Journal : Name of article in journal in italics / Name of journal not in italics
Volume of journal in italics
Word, "page" or "pg" not used. Just write the number range
Report should be (a) 3-4 pages
(b) Spacing - 1.5
At first I wondered what was the point of writing a project report as report writing was our first module and this module was on classroom learning environment and not report writing. It was only later tonight, when I was writing the rationale for the group report that I realized that the point was not to teach us how to write a report but the report was the assessment product to detemine if
(a) we had read enough on the topic - Lit review
(b) we understood the meaning of learning environments and the importance of assessing the
learning environment so that we can create one that is the most conducive to pupils learning
as it meets the pupils' needs - Introduction, Rationale
(c) we know how to assess a learning environment
(i) know the methodology for assessing the learning environment - Sample size,
Instrument and Procedure
(ii) know how to use the software, SPSS, to find the Cronbach's apha coeeficient and the
paired sample T Test and understand their importance - Findings, Analysis
A great way to test our understanding instead of giving us the traditional test.
REFLECTIONS ON SESSION 4
Objectives for Session 4
(a) Features of a learning environment assessment questionnaire
(b) Evaluate the school learning environment through case studies
(c) How to assess a classroom learning environment
(i) How to administer a survey on the learning environment
(ii) How to use a special template for recording and collating the data collected and to
calculate the means
(iii) How to use the software, SPSS, to analyse the data to determine if the scales in the
questionnaire are reliable using Cronbach reliability coefficient and find the means
Overview Of Lesson
We learnt how we can assess the learning environment using instruments/questionnaires. There are already validated instruments that we can modify to suit our requirements. We need not start from scratch and need not have to validate the instrument. We looked at one instrument, The School Level Environment Questionnaire (SLCEQ), in detail to learn the features needed to construct a questionnaire.
When constructing a questionnaire, we need to determine what scales we want to measure and ensure that they are aligned with Moo's schema.
In SLCEQ there are 9 scales.
Relationship - Student support / Affiliation
Personal development - Professional interest / Achievement orientaiton
System maintenance and system change - Staff freedom / Preparatory decision making /
Innovation / Resource adequacy / Work pressure
Each scale will be a block of an average of 4-5 items written in positive terms. There should not be too many questions in a questionnaire - 25 items is a good number. The Likert scale should have 4-5 responses that are arranged from the positive (strongly agree) to the negative (strongly disagree).
Analysis of the questionnaire using the software SPSS should be done scale by scale and separately for preferred and actual questionnaires for Cronbach's alpha coefficient.
We applied what we learnt by having a hands-on session. Everyone completed the preffered and actual questonaaire for WIHIC (What Is Happening In This Classroom), modified the data recording template given by Dr Quek to suit the questionnaire we used and analysed the data using the software, SPSS.
Reflections
Assessing the learning environment is very important if we want pupils to maximize their learning in our classrooms. The teacher's perception of her LE may not the same as those of the pupils. The only way to determine what aspectsof the learning environment are a problem and need to be adddressed through intervention measures is by getting feedback from the pupils, either qualitatively through a questionnaire with a Likert scale or qualitatively through interviews. Usually a combination of both is needed to identify the AFIs (areas for improvement) and to have a deeper understanding why those are as are identified as AFIs. However, there are two major problems with this :
(a) the teacher needs to know how to adminster such a survey and anlyse the results
(b) the teacher must be able to accept negative feedback and use it as a learning point for improvement instead of taking it as a criticism and feel hurt and diminished.
Where analysing the data is concerned, simple statistics need be used at the teacher's level. However, it is still possible that many teachers may find it difficult to overcome the barriers of having to key in so much data as well as to do statistics (it is an inhibiting word to many non-Mathematics persons). However, I still feel that even without the Cronberg's alpha coefficeint and t tests carried out, we can still get pupils to record their answers on the optical sheet (OTAS) and use the OTAS marker to collate the data. The percentages given by the OTAS marker software can be used to determine if an item is a strength or an area for improvement if we set a target eg. 80% and above is a strength and 60% and belwo is an AFI. uing this method will save time and thus result in more teachers being inclined to assess theri learning environment. I find using the SPSS software necessary when carrying out a research project like Action research.
Not many teachers are comfortable having pupils evaluate their learning environment. This year, in my school, teachers were encouraged to use pupil evaluation as a form of professional development. However, very few teachers chose to do it and the main reason was being confortable getting eedback from students.
However, it was good to learn that there are ready-made questionnaires for different learning environments that we need only to modify them to suit our situations. We can remove certain items or add a few items without having to validate the questionnaire. This will encourage teachers t be more inlcined to use surveys to assess their learning environments.
We also looked at three case studies to determine if the school had a positive learning environment. In determining these scales, we need to keep in mind Moo's scheme : Relationships (R), Personal development (PD) and System maintenance and system change (SMSC).
Cronbach alpha reliability -
did not understand statement
interpretation of statement
may not be culturally suitable
Cronbach Alpha Coefficient
Cronbach's alpha measures how well a set of items (or variables) measures a single unidimensional latent construct. When data have a multidimensional structure, Cronbach's alpha will usually be low. Technically speaking, Cronbach's alpha is not a statistical test - it is a coefficient of reliability (or consistency).
Cronbach's alpha can be written as a function of the number of test items AND the average inter-correlation among the items. Below, for conceptual purposes, we show the formula for the standardized Cronbach's alpha:
Here N is equal to the number of items and r-bar is the average inter-item correlation among the items.
All this sounds somewhat difficult to understand. So what does this mean?
If the inter-item correlations are high, then there is evidence that the items are measuring the same underlying construct. This is really what is meant when someone says they have "high" or "good" reliability. They are referring to how well their items measure a single unidimensional latent construct.
Reliability coefficient of .70 or higher is considered "acceptable" in most Social Science research situations.
Cronbach's alpha can be written as a function of the number of test items AND the average inter-correlation among the items. Below, for conceptual purposes, we show the formula for the standardized Cronbach's alpha:
Here N is equal to the number of items and r-bar is the average inter-item correlation among the items.
All this sounds somewhat difficult to understand. So what does this mean?
If the inter-item correlations are high, then there is evidence that the items are measuring the same underlying construct. This is really what is meant when someone says they have "high" or "good" reliability. They are referring to how well their items measure a single unidimensional latent construct.
Reliability coefficient of .70 or higher is considered "acceptable" in most Social Science research situations.
Subscribe to:
Posts (Atom)