Becoming A More Critical, Autonomous, Reflective Learner: Student Handbook
Becoming A More Critical, Autonomous, Reflective Learner: Student Handbook
Student Handbook
It has been compiled and edited by Mari Koivisto: University of Turku, and Leena Jokinen: Turku School of Economics & Business Administration
Project Contact:
Barbara Merrill, Centre for Lifelong Learning, Westwood Campus, University of Warwick, Coventry CV4 7AL UK Email: [email protected]
Index
1.2. 1.3. 2.2. 3.2. 4.2. 5.2. 5.3. 5.4. 5.5.
Index
Introduction .................................................................................................................... P3
. What is autonomous learning in higher education and why is it important? .......... P3 1.1. Theoretical framework underpinning this handbook Benefits of being critical, autonomous and reflective What do we mean by the term autonomous learner?
. (Self) Reflexivity Not only benefits, but also risks? ............................................... P0 2.1. Problems and risks of (self) reflexivity Possible way out: reflecting co-operatively a control strategy
3. Learning context ......................................................................................................... P0 3.1. Cultural aspects 3.1.1. Minorities / Majorities 3.1.2. Diversity, traditional students vs non-traditional students Learning environment
3.2.1. Supportive learning environment 3.2.2. Supportive environments 3.2.3. Guidance, interaction and dialogical learning 3.2.4. Students emotional experience of critical reflection
4. Some tools and activities to promote autonomous learning .................................... P0 4.1. Group activities 4.1.1. Group work 4.1.2. (Ethnographic) case studies 4.1.3. Use of role-play / simulation Personal activities what you can do by yourself
4.2.1. Learning log 4.2.2. Portfolio 4.2.3. How to tell/write your own story? 5. Methods of evaluation and assessment .................................................................... P0 5.1. Introduction Diaries / Learning logs Portfolios Peer evaluation Self-evaluation / Involvement
Introduction
This handbook is for students in higher education, that is, in a university or in another institution which gives the highest level education in the country. We have written the handbook especially for non-traditional students who enter higher education by alternative routes or are different from the student norm in other ways. This booklet helps to guide you to become a critical, autonomous and reflective learner. As a non-traditional student we mean A new mature student entrant (by age in respective countries) with no previous higher education qualifications whose participation in higher education is constrained by structural factors additional to age. We have met and interviewed several people in different countries who can be called non-traditional students, like a plumber who started his law studies at the age of fifty, or a hair dresser and a child minder who became a Russian language major when her own children grew up, or a female pig farmer who started to study economics, or a male electrician who studied to become a nurse, or a male shop assistant who first acquired a Social Studies degree, then went on to become a PhD, and now works as a university sociology lecturer.
to develop your ability to reflect and evaluate your learning to help develop more equal dialogue with teachers and fellow students to use your knowledge and prior experience as part of your learning to cope with the demands of higher education to get practical hints from students in similar situations to make your studies rewarding, creative and more personally involving
Your own role is vital in developing critical thinking, but you do not have to do it on your own. Institutions have systems and procedures which help adult students to foster critical reflection. Teachers and tutors are there to provide guidance and tools to develop your own thinking patterns and theories in action. Your peer students as well as student unions can also be very good source of support and help you to get most out of your studies. With the help of this handbook you should find out more about what critical reflective learning means. You can use this guide as a tool for bringing critical thinking into your studies. This handbook is mainly based on the research project we conducted in seven European countries during the years 2004-2006. The project name was Promoting Reflective Independent Learning in Higher Education (PRILHE).
identify the learning processes which enable (non-traditional) adult students in higher education to become independent reflective learners identify how this process could be better supported before, within and at the end of study examine the interface between learning from experience and academic learning the overlaps and conjunctions and how experience may help academic learning and future learning, including in the workplace identify models of good practice in higher education institutions across Europe produce resource materials for both adult students and lecturers.
PRILHE - Promoting Reflective Independent Learning in HE Student Handbook The project involved the following partners;
University of Barcelona (CREA), Spain University of Goettingen (Education Department), Germany University of Lower Silesia (Department of Education), Poland Universidade Nova de Lisboa (ISEGI), Portugal University of Stockholm (Department of Education), Sweden University of Turku (Centre for Extension Studies), Finland University of Warwick (Centre for Lifelong Learning), UK
Experiential learning
Critical learning
APEL (Accreditation of Prior Experiential Learning) Pragmatist tradition (Dewey, Mead) SWEDEN / PORTUGAL
Critical theory (Habermas) Social learning Collective learning Dialogical learning SPAIN / UK
At a later stage, the Swedish and Polish teams added a further review of Transformative Learning which complemented and, to some extent, bridged the three strands above. After all these sections had been drafted and discussed, it was agreed that the term, Critical Autonomous Reflective Learning (CARL) was a more appropriate key concept than the original idea of Reflective Independent Learning. We have defined CARL as follows:
critical in its awareness of the wider social conditions of learning autonomous in its understanding of the inter-relationship between learner dependency (on the one hand) and the potential for independent individual and group learning and action (on the other) reflective in the way it looks, from a certain distance, both at the learning process and the assumptions that underpin it .
Critical autonomous reflective learning thus links individual and social learning, seeing it as a transformative process which has the potential to change both individuals and social structures. This understanding seems to be particularly appropriate for the learning practices of non-traditional adult students in higher education.
I learn bestwhen I can determine the learning objectives, learning content and way of assessment self-reliantly or collaboratively (in a group) and when the learning environment supports this very self-directedness.
(Student, Germany)
What I like most here is that you can be very creative during classes, that you can say something wise on the basis of your own experiences or on the basis of read literature, you can show your creativity, you can create a theory which will be adopted by the others or which can be discussed. We do not base our ideas only on a concrete bibliography and literature we have to read, the truth still takes place on the basis of a free discussion and this is the nicest as we learn most from this.
(Student, Poland)
Practical experiences, for instance in working life, provide a solid ground for critical perspectives. Practical issues can offer you critical arguments when you reflect on a theory. In the same way, experiences can make more sense when you look them in the light of a theory. Practical implications and links to practice usually make learning easier and keep you interested and involved in a dialogue. Now that I have done work experience in various places I notice that I sort of think about my own actions more. And especially what are the good sides of an exam or a group work or something. So I do a lot of reflection all the time.
(Student, Finland)
Things must also make sense to the student: If things make sense to me I never forget them If I learn memorizing, the following semester I remember nothing.
(Student, Portugal)
In the development of critical, autonomous and reflective thinking, lecturers are also responsible for helping the student to develop such skills. Not all students are used to the fact that critical thinking is encouraged at school or university: At my first course the teachers said You should question everything. That made a great impact on me, because then I had to learn to think on my own and to analyse. In my home country the attitude is that the teacher is always right.
(Student, Sweden)
Although students complain that essays and project work (with research) imply more work than a final examination, they also recognise that through these activities the learning is more effective and they do not forget so easily the content learned. This means that helping students to learn in a more autonomous way will improve their capacity to become more critical and reflective. Naturally, it will also promote better and deeper learning. Here is a lecturers opinion: I think that it is of added value that there are older people in the classroom, because they contribute a level of reflection which is more involved, in order to specify more content or to give different levels () while at the same time they contribute to myself and to my colleagues a great deal of knowledge and interesting issues.
(Lecturer, Spain)
Interviewer: Have you changed as a result of your ethnology studies? Student: I suppose so. But that was a process which had started beforehand. That you simply begin to question
certain things that you have taken for granted, somehow. And - - that you begin to question certain moral concepts and things like that. So this had begun before and - - of course has been increasing during my studies. Well - - you just tackle, you realize more easily that it is just one view on matters. And - - that this very view definitely need not to be the only right and true one, even if you are often told so. I really think (speaking louder) that this process has been strengthened through these studies. But if it hadnt been there before, maybe I wouldnt have begun studying ethnology, because it wouldnt have interested me to break it open.
(Student, Germany)
My thinking has been transformed from rather simplistic one with a clear image of reality towards more critical thinking. Both lectures, seminars and my own work put together are functioning best.
(Student, Sweden)
Yes, you learn to see from other perspectives than just your own.
(Student, Sweden)
Interviewer: Has your approach to learning changed since you entered higher education? Student: I am lacking the vigour and enthusiasm to take the ideas in (cramming) uncritically (as done in the first
study course and as expected during the additional course of study). For me it is important to get an overview of the issue and to think about the possibilities of application.
(Student, Germany)
I studied philosophy, mathematics, education, administration, multimedia informatics, and now I study Higher Education. My approach is a trans-disciplinary one, with the integration of different patterns of thought.
(Student, Germany)
According to your opinion, how autonomous are you as a learner? What are the main advantages of being autonomous in your field of study? What could be the practical implications of being an autonomous learner?
At this point, it is by no means about questioning the benefits of reflexivity, but we have to remember some problems that may occur. So, what are the problems that may arise? Here is a German example of what can happen when you try to become more reflective. The midwives in hospital seem to be so constricted in their thinking, I find that very disconcerting. Sometimes I think a lot of them dont want to learn anything new. [] In discussion with a fellow student (who is employed and a student like me) I noticed that she has the same experiences with her colleagues. And I have similar problems with some friends of mine (midwives too); or have I been changing so radically, that Im looking for other levels? I dont know yet how I should/could handle my friends? At the moment I keep my distance; I feel the need to find myself, to find a course for me, but above all a strategy how to handle this situation.
(Student, Germany)
Reflection can turn in on itself and thus sometimes leads to a kind of action paralysis. Reflection can be socially risky when students become alienated from their social background as a result.
Social detachment
A disturbing and risky side effect of reflexivity is that you might become detached from your social contexts. As we can see in the above example, when you get preoccupied with and open up towards some challenging theories as well as confront unfamiliar and multiple perspectives during your studies, you may start to reconsider your earlier experiences, evaluations, beliefs and ways of behaviour which you have taken for granted until this time. If your social background is not flexible enough to accept and support these alterations in thinking of and dealing with reality, it might have a kind of alienating effect in two ways. You may not understand any more what is going on in your original context, you may become alienated from activities to which you or significant others (used to) attach high emotional importance and may no longer meet social expectations within this context ; on the other hand, the original reference group - facing these changes - is likely to experience a similar feeling of strangeness and they might let you know it quite clearly.
What do you think are the main disadvantages or risks in being an autonomous learner? What kind of practical obstacles have you encountered in an autonomous learning process? What could be the possible solutions for you to overcome the obstacles and manage the risks involved in autonomous learning?
0
3. Learning context
In this chapter we discuss the cultural issues which might effect your learning. For example, how belonging in a minority group and being a non-traditional learner in a majority group were experienced among the students we interviewed. We wish to show you how non-traditional students think of themselves as members of peer groups. As this handbook is meant for non-traditional students we hope the quotations here support your self-reflections and add value to the process of becoming a critical reflective learner.
Many adult students find that the age difference is most noticeable in seminars as one student remarked: But those girlsthey just made you feel thick. In the third year you do feel thick and you do feel that you shouldnt be here and you do feel old and feel really on show and I dont particularly like that.
(Student, UK)
And another student adds: In my opinion, it is good to have students of all ages (in a group). I have noticed that when Im walking in the corridors of my university, the younger students approach me rather easily and ask for advice like I was someone from the staff. They believe that I know everything and in practice its true Therefore, being older gives you another role that you are like a tutor or advisor to the other students I dont mind having that role. I understand that the beginners have situations where they dont necessarily know where to go and what to do.
(Student, Finland).
A German student also notes: Yes, and with the other students - - I noticed, that theyve finished school not a long time ago - -not very much responsibility. So, I would not describe myself as very old, inside of me not very old but you notice, when you have worked many years, and you know what it means to have respites and to meet ones deadlines - when you have held a little bit of responsibly.
(Student, Germany)
I was the eldest person in the seminar group and I think because I was my opinion was valuedthe seminar tutor always used to say how are your grandchildren. So not only was she interested in me as a student she was also interested in me as a person.
(Student, UK)
The younger students were probably quite nervous about me in the beginning and thought that theres someone whos going to tell them what to do, but I adopted a rather silent role. I let them to have their space because it was their learning opportunity. If I had started to tell a lot of stories the situation would have changed totally.
(Student, Finland)
Others find that although they might be in a minority as adult students they get on well with younger students and that both groups learn from each other: You know in our group there are younger and older students than me but here the communication is very good. We all get on well we all help each other, all of us lend each other notes, make copies, call each other, exchange materials, so there are no problems.
(Student, Poland)
Most adult students support each others learning through peer group support as working in a group together can help to overcome the feeling of being in a minority. It is important to feel comfortable in order to develop critical thinking and express yourself as a member of the minority group. As a Swedish student states: By creating smaller groups it can be possible to push forward more students. In smaller groups people can become encouraged. Everyone must dare to hear his or her voice. Everyone must dare to hear his or her opinion, I think, and dare to confront what their opinions are and to see which one is a prejudice.
(Student, Sweden)
PRILHE - Promoting Reflective Independent Learning in HE Student Handbook 3.. Diversity, traditional student vs. non-traditional student
The characteristics of traditional students differ from those of non-traditional students because they belong to different age groups. This means that they have probably followed different routes to arrive in higher education and have different objectives to attain. Usually non-traditional students work (or have already worked) and so they have some work experience. This means that they can more easily make the link between theory and practice. One can even say that they prefer practical classes or classes in which they can immediately see the practical application of what they are learning. Moreover, the fact that they pay fees also gives them the right to be more demanding of the lecturer. Some of these students have similar life stories: From my students tales emerges a specific pattern: many of them are women after maternity leave, who decided to have a baby just after graduation, then they did not work and at some point they decided to do something with their lives. Very often a kindergarten is a place where it is relatively easy to get a job for a woman, and when they succeeded they wanted to stay and needed further education as a condition to keep the job. So, even though they were somehow forced to study, some of them made huge progress while we were working together during seminars. It is a pleasure to look at them, how they are developing, Im much more pleased and satisfied with them than with the full-time students.
(Lecturer, Poland)
A traditional student often prefers to discuss with a non-traditional student in the classroom, instead of discussing with the teacher (maybe because they have similar problems and interests). In the context of the interviews carried out for the PRILHE project, some teachers identified that non-traditional students may employ different learning strategies in class. For instance, a non-traditional student may take no notes while the lecturer is speaking, because he/she is listening and making connections with previous knowledge. A Polish student adds: The main difference between those who have had a break and the young ones is the potential of life experience and one can see it during the classes, for example during the social policy course their professional and life experience is huge here, because when we discuss issues like taxes or unemployment they are able to talk about their own reality based on their experience, it is much more real to them than to those who only have heard about unemployment from the stories of their distant uncles or aunts. They dont have academic skills, so they participate in a different way; using their background, their activity is rarely based on papers and articles which Ive given them, but it is interesting for me to observe how they perceive what reality brings to them...
(Student, Poland)
These two kinds of students may also attend courses for different purposes. For instance, non-traditional students enroll on a course (for example, the Curso de Especializao Tecnolgica post-secondary education, in Portugal) usually for professional advancement, career progression and career reorientation. This kind of student is more demanding because he/she already has experience. They see the course as an opportunity to update what they already know. They bring their work experience reality to the classroom; they are a challenge for the lecturer as they want to obtain results. They are more critical because they are tired and they have left their family to attend classes. So, classes must be appealing, enticing and motivating. The lecturer must meet their expectations. In comparison with traditional students, they are more concerned with finding a job. Some students prefer attending classes to other learning. I dedicate myself a lot to the classes and I dont study outside it. I try not to miss classes and I keep focused in what teacher is saying (). I try to understand how the reasoning works instead of learning by heart (memorizing) formulas.
(Student, Portugal)
3
During their course students prefer less theory and more practice. Moreover, although non-traditional students have some difficulties, they also have more experience. They also tend to have great enthusiasm. They are rather critical because they have experience. They look for better and deeper explanations. They have an idea of how the exercise should be solved and so they do not accept straight away what the teacher says. The teacher must prove that what he/she is saying is correct. Traditional students are different from non-traditional ones as they tolerate strategies focused more on theory (instead of practice). Traditional students can be good students because they already have some good study habits, have more time to study, and are more dedicated. They interact more with each other to solve problems. However, they are immature. For some of them, school does not make a lot of sense. Some do not have objectives and so they are lost, distracted and can disrupt the class. They tend to use the library less than non-traditional students or even not at all. On the other hand, non-traditional students miss some classes, have not enough time, arrive late at classes, usually are already tired and generally are impatient. However, they are also engaged because they make a big effort to go to school after work. This means that a traditional student can be a better student because non-traditional students do not have so much time to study. However, those non-traditional students working directly with the subject being taught learn more easily. [I select the topic]. When I see how and where I can apply what I am learning, it is easier to learn. When I dont see how I can use that, it is very difficult to study and learn.
(Student, Portugal)
Some adult students feel that learning must be done during the working life and get benefits from that as often professional / vocational courses can be more interesting and usually more practical: Enterprises are always at risk and if we dont progress, the enterprise does not progress too. I have already learned a lot of things in the marketing course and have applied a lot of them in my enterprise. Classes are interesting when I can see how I can apply and change the way I work in my enterprise this is the advantage of the evening class student he knows the real life and has practice (and the traditional students do not have this).
(Student, Portugal)
Classes with both non-traditional and traditional students can be more enriching than those with only non-traditional or traditional students because the former can help the latter to learn through their experiences, histories and cases. For example, at the University of Warwick (UK) there are degree modules that enable adult students to talk about their learning experiences and compare these to research findings (Module on Adult Learning & Teaching, Module on Family Law). One lecturer in Portugal said that separating non-traditional from traditional students would not improve the learning process, as he could no longer use the experience of non-traditional students to help traditional students by making the bridge between the theory and practice. As for the relationship between non-traditional and traditional students, a British lecturer adds: When you have mature students and school leavers together I think it does raise particular issues about dynamics partly because mature students are more likely to intervene and have their say. They also bring in more varied agendas and the school leavers tend on average to be reticent. Often its because mature students are really engaging and bringing their life experience to bear with the material.
(Lecturer, UK)
4
A Portuguese lecturer presents the following viewpoint: In the last class I had to talk about security in networks and I had 3 or 4 people working with computers, doing some processes. And those students didnt know why they had to do what they do. So, I picked up those examples and asked everybody to say why they were doing what they were doing. And after that I explained. They realised that the topic I was explaining was not something theoretical without application. This is what the enterprise is doing.
(Lecturer, Portugal)
For non-traditional students it is important to explain how the things taught are related to the real world. After that the theory is explained. The order of topics to be taught can also change. That is how it can be shown that things are dynamic.
What do you think is the impact of cultural aspects on a critical learning process / environment? How do you think life experience could be best used in a learning environment? How could lecturers use your life experience to help you become more critical and autonomous? What do you feel about diversity in the classroom?
5
Institutional support
A study schedule (timetable) and its flexibility is one way to support learners. For learners who combine work and studying, time management seems to be an important issue. It is vital that an institution takes account of the restrictions learners have e.g. their ability to attend daytime classes if in employment or if they have children and cannot attend outside school hours.
6
As a student it is important to access e-journals and information on the WWW in order to assist you in your learning and assessment. Many university libraries offer training in how to use those facilities. I think theyre very supportive. They spend a lot of time doing tutorials and extra study skills support. The library staff are really supportive. I think they spend a lot of time over and above what were supposed to really.
(Lecturer, UK)
On entering higher education you may be anxious about writing your first essay (assignment) specially if you have been out of the education system for a long time. Many institutions provide special sessions on study skills support. These will cover topics such as reading skills, taking notes, essay writing, revision and taking exams. Departments are also an important infrastructure for creating a support learning environment. This includes both support from lectures and administrative staff. If you feel that the infrastructure and the staff are not facilitating your learning, make your voice known. Find out what your institution offers in terms of learning support and guidance. If you feel that the learning environment is not adequate why dont you raise the issues through your students association or staff student committees? Here is an example of institutional support at The University of Warwick, UK.
They are not more independent and reflective because we dont provide them with the conditions to be so. Moreover, they are not all the same. People need to be stimulated, motivated and that kind of learning must be learned.
(Lecturer, Portugal)
To create a sympathetic relationship between a teacher and a student requires efforts form the student as well. I warn them I say that we have to work but then comes the time when I warn them no longer. And that is it!
(Lecturer, Portugal)
Positive environment is important with a lot of encouragement from the supervisor as well as from the other students.
(Student, Sweden)
Monday 23.01.06, I met a fellow student after work, who is my age. We talked about our studies as we have already done quite often. We can help and support each other very well and I like being with her. Private and professional matters as well as issues around our studies are mingled. Our common study experience plays the leading role in our friendship. I like keeping in touch with her, because she knows all sides of my life and we can talk about it.
(Students learning diary, Germany)
I met my best friend in my first semester at the university. It has worked out very well in the first semester. And the funny thing about it is that her biography is nearly like mine, which means we had a common level from where we started. She also had problems with learning. And I have had the same personal subgroup with me since the beginning of my study. Of course some new people came to our group too and some left the group. But it was a circle of three, four, five persons, all with same subjects and related experiences. And we helped and encouraged each other. I would have been over-extended in my basic studies. Although we go different ways we pull each other along. And if you have a bad phase, a crisis, someone will be there to help you out. And we can encourage each other. Someone knows something, another one gives a good hint, and this has helped me a lot.
(Student, Germany)
Interaction and discussions with fellow students also form a set of resources which promotes reflectivity as well as fostering independence and critical thinking. Exercises and practices where students can exchange opinions and experiences in an equal and friendly atmosphere are seen to be very efficient for in-depth and quality learning.
There is a pedagogical approach that studies in depth the role of dialogue in learning: dialogical learning (Flecha 2000). Dialogical learning considers the main theoretical contributions in education and social sciences and the practical experiences recognised worldwide. Dialogue in the educational field is a way of assuring the democratic values and the transformation of exclusion situations (Freire 1997, Habermas 1984). On the other hand, dialogical learning does not forget technical knowledge and emphasises the role of instrumental dimension in education (Elboj et al. 2002) which includes the skills that are needed in order to obtain more complex ones later on. At the same time, the acquisition of these skills guaranties a continuance in the educational system, and also, the development of a high quality education. Dialogical learning believes in a curriculum of competencies that prepares students for the information society. As one lecturer states: And it might be that this experience motivates me nowadays to take individual students seriously, to approach some of them and to establish relationships to all of them, somehow. That is I support them in this very respect so that they do not have the impression to be left alone. They can get everything from me, they get when they come to me and say: I dont know and this and that, then I sit down for three hours and say: Let us think about what we do. Because I experienced how it is if you do not have this. In this traditional sense: My children shall be better off than me. , er I am doing differently nowadays. Er and with regard to the structures being responsible for the faculty - I see to it that there are supervisors, tutors, that there are supporting systems and something like that.
(Lecturer, Germany)
A student tells about learning with peers: At the Christmas market in the evening I explained my paper on motivation to a fellow-student. The one-hour-talk about this is a kind of reflection for me. Such discussions in pairs are consistently very useful for me.
(Students learning diary, Germany)
Interaction in teaching and learning is needed: In my case, during the weekend I have my wife, my 2 daughters and the baby born recently. Sometimes, there are also enterprise problems [requiring attention from me]. () there could be group work but completed during classes for instance, reserve half a hour (of a 2 hours class) to do the exercises / group work. How else can I find time to meet my colleagues to do the work?
(Student, Portugal)
0
Lecturers could also meet students outside class or answer their questions by email.
(Student, Portugal)
The role of the lecturer is very important. He or she has to be motivated and enjoy his/her job. Some lecturers just do not have the necessary pedagogical skills: They can be very good in their profession but do not know how to teach. Maybe there is a pedagogical component missing in their training. () All of them should do some trainee training. () I think that is awful being a teacher just because they cannot find a job elsewhere. I understand that it is in the universitys interest not to be too close to the real world; even so the professional must have the necessary skills to teach.
(Student, Portugal)
Lecturers also need to be demanding enough: We were scared, probably because we were not used to that level of pressure. Maybe it was the first time that I was forced to reflect and when I arrived at the university it was not hard for me to understand and accept the level of exigency of lecturers.() [we did] several essays, we had objectives, we did individual and group work, a lot of work every week. We complained, saying that it was too much but she replied if you managed till now that is because you can. You must know what you want. She was a demanding kind of person. At that time, we complained but now I recognize that she was right and that kind of work, organisation and research is very important.
(Student, Portugal)
Before starting studying he had noticed the same development with a friend who studied at higher education. I noticed this the first time when a male friend had become more tedious through studying. We were often taking a beer together and earlier he always accepted what I said. After he started to study he also started to question me. It was terribly unpleasant. He always wanted to know more, irrelevant facts in my opinion. My friend was earlier a storyteller as myself but he had given up to tell stories and instead become critical. I noticed how he was changing.. you probably have to say that he developed. But I experienced it for a long while as a negative change but of course it is development.
(Student, Sweden)
To face this change or development might be an emotional experience that is not always pleasant. Rather it is impregnated by contradictory emotions, where the individual sometimes feel stimulated, happy and safe and sometimes feel unhappy, guilty and frustrated. To develop critical autonomous reflection is therefore also an emotional journey that must not be neglected or underestimated. Brookfield has given experiences like Karls a name that is a little bit brutal but also illustrative; cultural suicide, Cultural suicide is the threat critical learners perceive that if they take a critical questioning of conventional assumptions, justifications, structures and actions too far they will risk being excluded from the cultures that have defined and sustained them up to that point in their lives.
(Brookfield 1994, p. 208).
To a student this can mean becoming a foreigner in his or her own family or together with his/her old friends. The things they were laughing about together before is maybe not any longer commonly appreciated and a gap between them might be felt by one of them or maybe by both of them. Marriage might end up in divorces and friendships might fade away. One important way to handle those emotional aspects of reflective learning is to share it with other students in communication. When talking to each other, students discover that they are not alone experiencing those emotions. This discovery is often relieving to them and to put words on what is difficult can play down some of the difficulties experienced. Another connected concept by Brookfield (ibid.), Lost innocence, describes education as a journey into ambiguity and uncertainty. The belief in one truth is abandoned as well as a dualistic thinking by the learner. This personal development is well described in many of the interviews and often it is experienced as a positive development1. Here is a German lecturers viewpoint: I would say you can distinctly observe here, that especially mature students, even the postgraduate students, face massive biographical problems they have to cope with and that you cannot really call this study. Simply due to the fact that they are studying here something is stirred that then is messed up and so they have to reconcile all this somehow. Of course this works only by means of an increased theoretical effort that you reflect upon and that then allows a new view on the things. In so far it is not that easy if you are studying here at an older age. I myself experienced this and have to admit to myself that this was my greatest problem. If you really let this happen to you, you have to alter attitudes and opinions showing through biographically. And then you are having a big problem, since you do not only have to learn this about yourself, but you really have to change yourself and maybe to alter your whole idea of your life. And this is not too easy.
(Lecturer, Germany)
What are your own supportive environments and resources? How do you see your role in your study group, what does the feeling of belonging to the group mean to you? What can you do to help others feel as part of the group?
In the PRILHE literature review Brookfields concepts are described more in depth.
The group and the team sort of draws you in. As its quite sad to work alone. Even though you had the motivation to study you are still alone.
(Student, Finland)
The compilation of the groups can have an effect on the learning. Here is a comment from a Finnish lecturer: The students find it very important to learn from each other during a course. Therefore, its important that when we plan a course we take students who, at one level, are heterogeneous and, at another level, homogeneous. Homogeneous in a sense that they can find other people who are in a same position, such as managers or experts. And heterogeneous in a sense that people come from different lines of business so that they can see different points of view.
(Lecturer, Finland)
In some situations group work is carried out as a single work task. However, this is not what group work is about. As one student points out: Rather often group work turns out to be an amount of individual work that is put together.
(Student, Sweden)
3
The interactive groups achieve two goals. The first one is related to being and learning together, because students of different cultures, gender and academic levels learn to help each other, and this is a way to promote solidarity and open dialogue. The second one is related to instrumental skills because thanks to interactive groups the learning process is to improve or strengthen learning and students acquire more knowledge. But otherwise we have less moments of discussions or special lectures to work reflexively about this. Mrs K. offers this course about writing and speaking. In principle this is a reflexive course. Indeed, it is specific, but it helps us to realise that there are different modalities of speaking, and you can practice them, especially for your own subject, and that there are different modalities of writing. She holds this course for the second time and I think its well accepted. Sometimes students tell me they would like to have such a course more often because firstly it is fun, and secondly it gives you the feeling of: how to evaluate your competence? How to advance?
(Lecturer, Germany)
Attending classes is a very important issue for success. It is like a ritual: It is more dynamic. We arrive here already tired after a long working day. If I dont participate, after a while I do not pay attention any more. I am almost asleep. If I participate it is different. I am more attentive and memorise better. I remember a class where the teacher asked us questions and it is us [the students] who defined concepts following the questions asked. She asks a lot of questions. Intrinsically, we know the topic (). With the questions, the teacher helps us to conceptualise the topic and at the end she puts the names to what we try to express
(Student, Portugal)
4
Case presenter: S/he describes a situation s/he experiences as difficult. Here, s/he focuses on the external circumstances and on his/her feelings, viewpoints, coping strategies as well. Moderator: S/he conducts the session. Counsellors: The remaining participants bring their difficult perspectives, experiences and competences as resources in the process and hence try to encourage the case presenter in gaining new perspectives: You come more easily to a modified perception of yourself through this kind of outside perspective and through appropriately given feedback; it facilitates a versatile and multilayered consideration of your own ways of acting, your own opinions and the responses which arise from this among interaction partners. By this, scopes for decisionmaking and action are extended and the participants are enabled to cope with difficult situations on their own.
(Tietze 2003, p.22)
Process observer: S/he gives the group feedback about the consultation process. This serves as a means for quality assurance.
From session to session the different roles are allocated to other participants so that each one is enabled to fill in all roles with its respective responsibilities.
5
Phases Casting
(5 minutes without the warm-up round, 45 minutes including warming-up round)
Warm-up round if necessary Selection of a moderator Course of practice is discussed, additions and agreements are made if necessary Participants present theirs requests Additional role allocation: Selection of a case presenter, the remaining participants are the consultants Defining the order of the supervisions
Spontaneous What is it all about? How does the case presenter experience the situation? Case Presentation
(5-10 minutes)
Organised as a conversation between case presenter and moderator The case presenter sketches his/her problem on the basis of a concrete situation / key scene The peer consultants listen to the narration attentively without interrupting The moderator supports the case presenter in his / her description activates experiences and ideas by asking questions The consultants may ask questions for a better understanding at the end
Key Question
Which question/need has the case presenter with regard to the situation s/he is in?
Method Choice
5 minutes)
The case presenter puts his/her key question supported by the moderator Consultants make suggestions how to put the key question if necessary
Which method?
Dialogue between case presenter, moderator and consultants Selection of a consulting method from the method pool (rather solution-oriented, sympathetic or geared to an alteration of perspectives)
Consultation
(10 minutes)
What do we tell the case presenter with regard to his/her key question?
Consultants put ideas and suggestions according to the style of the chosen consultation module
Conclusion
(5-15 minutes)
What did the case presenter learn from the peer group supervision?
Case presenter sums the contributions of the consultants up and states which suggestions seem helpful/not helpful to him / her Planning of initial measures if necessary Evaluation on the part of the consultants Feedback to the moderator Feedback of the process observer
6
PRILHE - Promoting Reflective Independent Learning in HE Student Handbook 4... (Ethnographic) case studies
Case studies can help you engage actively in the learning process. They are a useful tool to help you understand the theoretical and conceptual issues in an applied situation. Fry et al (1999) define case studies (as used in higher education teaching) as complex examples which give an insight into the context of a problem as well as topics that demonstrate theoretical concepts in an applied situation. According to Mustoe and Croft (1999) the use of case studies in teaching increases the motivation and interest of students.
Allow the application of theoretical concepts to be demonstrated, thus bridging the gap between theory and practice Encourage active learning Provide an opportunity for the development of key skills such as communication, group working and problem solving Increase the students enjoyment of the topic and hence their desire to learn.
Our research and experience tells us that adult students often like to have a practical reference point or context in which to make sense of theory. Relating back directly to personal experience can be productive but it also raises problems about how this experience should be used. For example, some students can become trapped in their own experience to the extent that they do not question their underlying assumptions and cannot see other perspectives clearly. In this context, the use of ethnographic case studies can be a useful approach. It helps give students a practical context in which to develop their ideas but avoids being focused exclusively on one particular experience. It also helps students to get different perspectives on key issues. I think trying to get people to find material which is close to them in some ways in their experience and yet perhaps challenges some of their assumptions its no good just representing another point of view in the abstract or from a long way away. In the Sociology of work for instance theres a whole range of quite rich ethnographic case studies. I think these can be very fruitful for mature students to engage with. Theyre entrenched as it were. Its very difficult for them to step back. An obvious truism is that mature students vary tremendously in both their areas of interest and their abilities. Engaging with their experience is not all straightforward and positive.
(Lecturer, UK)
For a student, the role-play implies that you have to improvise and be creative with what you have learnt in real situations. Conflicts might occur and interaction situations in the cases become more real to the students when acting.
I think that these individual reflections are a good idea. They helped me to acknowledge what I learned and how my learning evolved this semester.
(Student, Portugal)
I have now written a learning log more regularly than before and made comparisons between the issues that I have learnt on my course and my working life and tried to find the things I could use in my jobI dont know if that is of any use in my work but I have many ideas.
(Student, Finland)
Once a month I have one contact teaching day with a group of adult students. Its very informal, sort of very free. Its based on the fact that the group really does a lot of studying. We go through their learning logs, they write, the learning logs are based on themes and teaching theories and so on. But in my opinion, it entails different kind of freedom and confidence in the fact that the students study when they are able to set their own goals.
(Lecturer, Finland)
You may initially find the diaries and logs difficult or meaningless, but in the course of time you might find it very useful for your learning as well as for your abilities to be critical and aware. Furthermore, the fact that you have to write about new information or ideas enables you to better understand and remember the topic, and that articulating connections between new information or ideas and existing knowledge secures and extends learning. ()() When the learning journal [or log] is an active, experiential self study aid, it can help students move from surface to deep learning both within and across modules.
(ORourke 1998, p. 404).
As one Portuguese student explained: This log helped to reflect and analyse our work, what we did in the classroom, our work method and the value and importance of the subject for us and for our future.
(Student, Portugal).
Honey (000) suggests the following questions to be answered in the learning log:
1. 2. 3. My description of what happened My conclusions / lessons learned My plan to do something better / different
Have you ever filled in a log? What is your feeling about this approach to learning as a form of assessment? Did you find it difficult to get started? If you have experience of writing learning logs can you give any advice to your colleagues so they can start it?
http://cuip.uchicago.edu/schools/gearup/chicago/archive/2005-06/gu-avid/docs/log.htm
30
What can be included in a portfolio? Everything that is crucial or significant to your learning and development:
Questions raised while studying some theories, reading articles, discussing with peers and teachers. Individual comments on texts, lectures and any kind of class activities. Any paragraphs and quotations you have found relevant, inspiring, meaningful or - on the contrary controversial. Any visual forms of your ideas and issues, like graphs, schemes, icons or photos with comments explaining their importance and meaning for your development. Your own reflection on how you are changing and why you find this particular direction the most suitable for you. Ive finally understood what it means to learn. Thanks to working on my portfolio I started to think about myself how I am, what do I know, what I am comfortable with... and compare with other people. Ive noticed the differences between people and Ive realised that even the smallest event or trivial situation in my life is a potential source for reflection.
(Student, Poland)
According to the Quality Assurance Agency of UK, a portfolio is intended to help students to
become more effective, independent and confident self-directed learners understand how they are learning and relate their learning to a wider context improve their general skills for study and career management articulate their personal goals and evaluate progress towards their achievement encourage a positive attitude to learning throughout life improve and encourage dialogue between learners and teachers.
4..3. How to tell/write your own story? What does it mean to tell a story?
Telling seems to have a familiar ring: Fairytales are told. Grandmother tells a story. When we compare the sound of the word with saying or reporting, we notice that telling is more emotional. Telling is a way of reliving the past. This is what makes it so interesting. But the form of telling is also something rather special. Telling follows different rules from those of reporting or arguing (Kallmeyer & Schuetze 1977). If we get the chance to tell something to other people, if a circle of listeners or even a single listener gives us his/her undivided attention, then we must also have something to tell. We join in a kind of contract and have to do our bit to make sure that this contract is held.
3
from where you come and which obstacles hindered your educational career, which particular life experiences led you to higher education, which plans and dreams you have and which ideas may be not that realistic, which opportunities you should reflect on and which resources in yourself should be strengthened.
You are an autonomous individual, however, you experience this particular autonomy if you confront it with the autonomous experiences of others. Learning difference makes you accept difference and, at the same time, also accept yourself as a learner. You are an important person among others. Telling your story gives you the feeling: that is true. What makes it useful to write your own story? Of course, you are not a writer (maybe, not yet). However, to write down your own experiences, to reflect on the way you have gone, the problems you have solved, the people you have lost, new friends you have gained this is part of an extremely important learning process. Fix it, just for yourself. It will be more than a portfolio of recently achieved qualifications and certificates. It is like a treasure for reflections-to be. Dont be afraid of it. Actually do it. Telling and writing about yourself is the best way to get an idea about who you are. This idea will change over times which is normal and okay. But you will understand why it changes and why the changes are legitimate. We are lifelong learners. It is good to know not precisely what, but why and how we have learnt.
3
Remember the situation when you last told about your life. What prevents you from telling your story to co-students? Is there a tutor you would like to talk to and explain who you are? Try to write down your life experiences. Be very honest about it. Its just for yourself.
With continuous assessment, the student receives continuous feedback which will enable him/her to see his/her progress towards the objectives. As a matter of fact, one student said that: With continuous assessment no one can leave anything behind
(Student, Portugal)
Many students prefer this because: There are a series of smaller objectives, instead of one big one at the end of the semester .
(Student, Portugal)
Many students feel more motivated and interested in their work with continuous assessment compared to exams: Last year I had a final examination in English. I never went to a class. I only did the test and I have the feeling that I learned nothing. I didnt attend classes. I studied grammar alone. I prepared myself for the examination. But I didnt speak English, I didnt participate, I didnt hear, I didnt discuss. Now that I have continuous assessment I attend classes, I participate. I have the feeling that I learn and I am more interested .
(Student, Portugal)
33
There are several forms of assessment, e.g. essays, assignments, examinations, oral presentations, etc., each of them contributing to the development of different skills. In this particular case essays and dissertations will equip you with a higher level of critical reflection and autonomous skills even if it demands more of your time. These skills will be useful to you in the future.
Which type of assessment do you prefer? Why? Have you ever been assessed in a continuous way or by a final examination? Did you feel the difference in how you learnt and the skills you have developed? And if so, in what ways?
Formative assessment is a self-reflective process that intends to promote student attainment (Wikipedia, 2006).
34
PRILHE - Promoting Reflective Independent Learning in HE Student Handbook Questions for Discussion / Reflection
Have you ever filled in a log? What is your feeling about this approach to learning as a form of assessment? Diaries may involve personal reflection. How would you feel about this being used as part of your assessment?
You can see both the progress and the area you still need to work on.
You start valuing your own knowledge and using it in constructing new ideas
You become more self-confident in taking up discussion with your tutor, colleagues etc. in stating your own opinion and in thinking critically.
You can notice the links between your formal educational activities and everyday life practice.
You can raise new questions and demands about the process of learning towards your institution. You can make some significant points about your preferable way of learning.
You can get a new perspective and improve your reflexivity. You can see the importance of events in your life course and their influence on your identity.
35
PRILHE - Promoting Reflective Independent Learning in HE Student Handbook Fenwick (6) suggests a series of steps to follow when beginning to use portfolios:
Decide first what is to be collected, by whom and when?. Spend time introducing portfolios to learners. Work with learners throughout the portfolio process. Hold periodic conferences with learners. Grade portfolios holistically. Spend time responding to learners.
Fenwick (1996) also suggests a format for holistic grading portfolio. It would include themes such as a) preparation of portfolio, b) documentation and growth, c) evaluation of selected items and d) quality of reflections / self evaluation demonstrated at closing conference. For each theme there will be some statements and the assessment would be made using a Likert scale of 5 points. (For more information please look at the Tara Fenwick paper available at http://www.ualberta.ca/~tfenwick/ext/pubs/print/portfolio.htm)
Have you ever prepared a portfolio as part of your assessment process? If so what were the difficulties in doing it? What did you learn in this process? How would you advise a colleague who is preparing one? If you have not prepared yet a portfolio do you know how to start one? If you dont, do you know how to look for information to start one? In which areas do you find it could be useful?
36
Fenwick () suggests an evaluation guide comprising a series of questions for each member of the group / student assess the cooperation in the team. Each question would be rated between (Weak) and 4 (Outstanding). Examples of the questions are:
all members shared their ideas freely we offered support and encouragement to each other we tried to explore alternate ideas before we settled on a solution
(For more information regarding peer assessment in groups please look at Tara Fenwick paper available at http://www.ualberta.ca/~tfenwick/ext/pubs/print/peereval.htm)
How do you feel about assessing another colleague and / or being peer assessed? What qualities and skills would you need to assess another colleague or being peer assessed? What might be the advantages and disadvantages of this type of assessment?
3
Do you think you would find it easy / difficult to do this? Why? What kind of skills would you need in order to undertake self-evaluation? What do you think about self-evaluation as a form of assessment?
3
Experience and education: The 60th anniversary edition. Learning by Expanding: An Activity-Theoretical Approach to Developmental Research. The Theory of Communicative Action. The Philosophy of the Act.
Chapter 3
Interaction in learning processes
Bruner, J. (1996) Rogoff, B. (1994) Rogoff, B. et al. (2001) Scribner, S. (1988) Vygotsky, L.S. (1986) The Culture of Education. Developing understanding of the idea of communities of learners. Mind, Culture and Activity, Learning Together: Children and Adults in a School Community. Boston: Harvard University Press. 1(4), 209-229. San Diego: University of California. New York: Oxford University Press. Teachers College, Columbia University. National Center on Education and Employment (ERIC Document Reproduction Service N CE 049897). Cambridge: MIT Press. (Original published in 1934).
Dialogue
CREA (2006) Spanish National Report. Analysis of the Interviews. PRILHE Project. Promoting Reflective and Independent learning in Higher Education. Socrates Programme. Brussels: European Commission. PRILHE Project. Promoting Reflective and Independent learning in Higher Education. Socrates Programme. Brussels: European Commission. Barcelona: Gra. Lanham: Rowman & Littlefield Publishers, Inc. (p.o. 1997). New York: Continuum Pub Group. Boston: Beacon Press.
Comunidades de Aprendizaje. Transformar la educacin Sharing Words. Theory and Practice of Dialogic Learning. Pedagogy of the Heart.
Habermas, J. (1984) The Theory of Communicative Action. Vol I, II, and III.
Chapter 4
Interactive groups
Elboj, C. et al. (2002) Comunidades de Aprendizaje. Transformar la educacin. Barcelona: Gra.
3
PRILHE - Promoting Reflective Independent Learning in HE Student Handbook (Ethnographic) case studies
Davis, C. and Wilcock, E. (2006) Fry, H., Ketteridge, S. and Marshall S. (1999) Mustoe, L. R. & Croft, A. C. (1999) Teaching materials using case studies. In Materials Education. 12 Guides for Lecturers [online]. A Handbook for Teaching and Learning in Higher Education: Enhancing Academic Practice. Motivating engineering students by using modern case studies, International Journal of Engineering Education, UK Centre for Materials Education. Available at: http://www. materials.ac.uk/guides/casestudies.asp [Accessed 15 September 2006]. Glasgow: Kogan Page.
15(6), 469-476.
Learning log
Baker, H. (2003) Teaching tip: the learning log. Journal of Information Systems Education [online] Incorporating Peer Assessment in Adult Education [online]. Early version of the article Peer Assessment Without Tears: Making it Work, Journal of the Alberta Association of Adult and Continuing Education, May. Learning Log A Way to Enhance Learning from Experience. General Study Skills [online] 14(1), 11-13. Available at: http://findarticles.com/p/articles/ mi_qa4041/is_200301/ai_n9202415 [Accessed 15 September 2006]. Available at: http://www.ualberta.ca/~tfenwick/ext/pubs/ print/peereval.htm [Accessed 15 September 2006]. Berks: Peter Honey Publications. Murdoch University. Available at: http://help-online.murdoch. edu.au/students/studyhelp/l_log.html [Accessed 15 September 2006]. Masterminds Publishing, LLC. Available at: http://www. calhoun.k12.al.us/makes%20sense/Adobe%20Reader/DO% 20NOT%20OPEN%20program%20files/ Lesson%20Structure/BACK/Learning%20Logs.pdf [Accessed 15 September 2006]. Chicago GEAR UP Alliance. Available at: http://cuip.uchicago. edu/schools/gearup/chicago/archive/2005-06/gu-avid/docs/ log.htm Accessed 18 September 2006].
Fenwick, T. (1999)
Learning Log Lorenzi, F., MacKeogh, K. and Fox, S. (2004) ORourke, R. (1998) The learning log
Questions [online]
Preparing students for learning in an online world: an evaluation of the Available at: http://www.eurodl.org/materials/contrib/2004/ student passport to elearning (SPEL) model, European Journal of Lorenzi_MacKeogh_Fox.htm Open, Distance and E-Learning [online], 2004/I. [Accessed 15 September 2006]. The learning journal: from chaos to coherence. Assessment & Evaluation in Higher Education, . In An Introduction to WAC Writing Across the Curriculum [online]. WAC Clearinghouse. In Student/Parent Resources [online].Advancement Via Individual Determination (AVID). 23(3), 403-413. Available at: http://wac.colostate.edu/intro/pop5h.cfm [Accessed 15 September 2006]. Available at: http://www.avidregion4.org/resources/documents/student_ parent/LearningLogQuestions.pdf [Accessed 15 September 2006]. Available at: http://findarticles.com/p/articles/mi_qa4041/is_ 200307/ai_n9282189 [Accessed 15 September 2006] Available at: http://www.lifetimetraining.com/leadership-andfolder/your-learning-log.php [Accessed 15 September 2006].
Wagner, C. (2003)
Put another (B)log on the wire: publishing learning logs as weblogs. Journal of Information Systems Education [online], July 2003.
Portfolio
Fenwick, T. (1996) Assessing Adult Learning Using Portfolios [online]. Paper presented at the Atlantic Teaching Showcase Conference, October 1996, Universite St. Annes, Nova Scotia Available at: http://www.ualberta.ca/~tfenwick/ext/pubs/ print/portfolio.htm [Accessed 15 September 2006]
40
The Higher Education Academy. Available at: http://www.ukcle.ac.uk/resources/reflection/teaching.html [Accessed 15 September 2006].
Storytelling
Alheit, P. (1992) The Narrative Interview. An Introduction. Copenhagen (Ms.). Zur Konstitution von Kommunikationsschemata der Sachverhaltsdarstellung. Exemplifiziert am Beispiel von Erzhlungen und Beschreibungen, In Wegner, D. (ed.) Gesprchsanalysen. Vortrge, Hamburg: Buske. gehalten anllich des 5. Kolloquiums des Instituts fr Kommunikationswissenschaft und Phonetik, Bonn 14 16 October 1976 (pp. 159-274). Eine interaktive Funktion von Erzhlungen. In Soeffner, H. G. (ed.) Interpretative Verfahren in den Sozial- und Textwissenschaften (pp.104-126). Kognitive Figuren des autobiographischen Stegreiferzhlens. In Kohli, M. & Robert, G. (eds.) Biographie und soziale Wirklichkeit. Neue Beitrge und Forschungsperspektiven (pp.78-117) Stuttgart: Metzler.
Quasthoff, U. (1979)
Schuetze, F. (1984)
Stuttgart.
Chapter 5
Diaries / Learning logs
Wikipedia. The Free Encyclopedia Assessment [online]. Available at: http://en.wikipedia.org/wiki/Assessment [Accessed at 18 September 2006]. Universit de Paris-Dauphine 21-22 April 2005 Paris, France. 3(2), 117-122. Available at: http://www.ejbrm.com/vol3/v3i2/v3-i2-art3-friesner.pdf [Accessed 18 September 2006].
Learning Log Analysis: Analyzing Data that Record Reflection and Friesner, T. and Hart, Experiential Learning. Paper read to the 4th European Conference on M. (2005a) Research Methodology for Business and Management, Friesner, T and Hart, M. (2005b) Learning Logs: Assessment or Research method?. The Electronic Journal of Business Research Methodology [online],
Portfolios
Fenwick, T. (1996) Assessing Adult Learning Using Portfolios [online]. Paper presented at the Atlantic Teaching Showcase Conference, October 1996, Universite St. Annes, Nova Scotia. Available at: http://www.ualberta.ca/~tfenwick/ext/pubs/ print/portfolio.htm [Accessed 15 September 2006].
Peer evaluation
Incorporating Peer Assessment in Adult Education [online]. Early version of the article Peer Assessment Without Tears: Making it Work. In the Journal of the Alberta Association of Adult and Continuing Education, May 1999. Also available in Fenwick, T. and Parsons, J. (2000) The Art of Evaluation: A Handbook for Educators and Trainers. Toronto: Thompson Educational Publishers. Available at: http://www.ualberta.ca/~tfenwick/ext/pubs/print/peereval. htm [Accessed 15 September 2006].
Fenwick, T. (1999)
Self-evaluation / Involvement
Snchez, M. (1999) 69(3), 320-335. Available at: http://www.edreview.org/ La Verneda-Sant Mart. A school where people dare to dream. Harvard harvard99/1999/fa99/f99sanch.htm [Accessed 18 Educational Review [online] September 2006]. School of Adult People of Verneda St. Mart [online]. Available at: http://www.edaverneda.org/indexen.html [Accessed 18 September 2006].
4
4
43
UNIVERSIDADE