8
Impact on quality assurance, monitoring and evaluation
This section reports on the findings relating to the impact of
networked learning on quality assurance, monitoring and evaluation.
8.2 Quality of teaching materials
Where several people spoke about quality they referred
to the quality of the teaching materials developed. Some differences
emerged
between HE and FE institutions in relation to monitoring quality
of the development of materials as show in Figure 8-2 below. Three
of the HE institutions refer to teams or special units that contributed
to the quality of materials, whilst nine of the FE colleges refer
to such support for development. The majority of these colleges
have a unit or central team that helps with production of materials.
As can be seen from the discussion of questionnaire
responses there is evidence of change to quality procedures. The
interview data
does not fully support the questionnaire responses. This does not
necessarily mean that the quality procedures in those institutions
that have not made indicated that they have made major changes
are inadequate. The differences could simply be interpreted as
reflecting a lack of discussion of this topic within the interviews
rather than lack of such procedures. In addition, several of the
respondents suggested that their current quality assurance procedures
are sufficiently robust and do not require any changes to cope
with what is seen as an additional mode of delivery. This view
can be supported by the fact that the majority of institutions
did not see networked learning as replacing traditional teaching;
rather it was seen as an additional tool that could be used alongside
other forms of delivery in a ‘blended’ mode (see Section
7.1). It is worth noting however that in the HE institutions where
networked learning is being used for distance delivery new procedures
have been developed for monitoring the quality of these materials.
The interview data and questionnaire data in relation
to monitoring the development of networked learning materials are
largely in
agreement. The questionnaire data shows that 75% of FE colleges
and 43% of HE institutions indicated that major changes have
taken place in this area, showing that there is a considerable
contrast
between FE and HE.
Within HE, there was a stronger emphasis on the
need to ensure academic ownership of the development process. There
were some
interesting comments from individual respondents focussing on the ‘visibility’ of
the learning materials in networked learning compared to the traditional
classroom, as shown in Figure 8 3. There appears to be an acceptance,
particularly in HE, that putting handouts and lecture notes online
in the form that they had been developed for face-to-face delivery
the best way of engaging with academic staff, especially those
that might be reluctant to become involved. In addition, issues
around visibility may also be affected by the extent to which access
to a particular VLE is password protected. In some cases, the resources
that are within the VLE are only accessible to the lecturer and
the students who are studying on the module. This allows for greater
ownership and less threat in terms of materials being seen more
widely. However, it is notable that the visibility of the materials
has highlighted issues in relation to copyright and that more stringent
controls are in operation, at least in some institutions (eg H10).
The comments summarised in Figure 8-3 reflected
three different issues: the impact on academic staff and students,
the impact on
quality procedures and the impact on administrative systems.
In the HE institutions three respondents commented that there was
better sharing of information, for both staff and students, and
in FE institutions two respondents mentioned the easier access
to student information while another said that staff were now
more
accountable. The majority of comments, six from HE and four from
FE institutions dealt with quality procedures, and the greater
need for monitoring of quality of teaching materials. Finally,
there were three references from FE institutions to the need
for more frequent updates to administrative materials.
Three respondents noted that this form of learning
required closer monitoring of students in terms of the quality
of the websites
that they were accessing and to ensure that there was no plagiarism.
It was also noted by one respondent that networked learning allowed
for better standardisation, for example, in terms of the use
of common assessments.
| |
Monitoring of development of materials |
| H1 |
Two respondents noted that the development of materials was
not yet monitored but suggested that a need existed; currently
responsibility of individual academics. One respondent mentioned
a sub-committee that vets programmes for E- and distance learning. |
| H2 |
One respondent commented that they have now begun to think
about quality assuring individual lecturers' teaching materials
because of the greater visibility. Considering implementing
a Rapid Development Taskforce approach to course/materials
development. |
| H3 |
Responsibility of individual academics. Have asked for a
new materials development unit to be created. |
| H4 |
Four respondents noted that standards are important to this
institution; materials and staff development regarded as priorities.
Central support for materials development with an emphasis
on quality. |
| H5 |
One respondent and the questionnaire data suggested that
networked learning had not had any major impact. Individual
academics and departments responsibility for quality of materials. |
| H7 |
Development of distance learning materials is closely monitored;
however, these procedures are not yet fully developed for networked
(blended) learning. |
| H8 |
Development of fully online (distance learning) materials
closely reviewed; however, this process does not cover majority
provision from whole institution. Existing procedures seen
as robust. |
| H9 |
Documentary evidence notes distinction between distance and
open learning; special quality assurance procedures exist for
online materials for distance learning. |
| H10 |
New quality procedures developed to ensure quality of materials;
different stages of assuring content and peer reviewing it.
The 'learning experience' is also now monitored and external
examiner has access to online materials. |
| F1 |
Three respondents noted that the quality rules stated that
all learning materials be put into core pack for start of session
and that this is then put on intranet. Supported by central
unit. |
| F2 |
Three respondents noted the intention to ensure quality by
allowing for gradual development with all materials being checked
for quality. Separate central e-learning unit to develop materials. |
| F3 |
Academic staff are supported in development of teaching materials
by a central team but no central monitoring of quality. |
| F4 |
Two respondents mentioned the importance of having high quality
materials on view (development supported centrally); one of
the college web-sites is an award winner. |
| F5 |
Two respondents noted that there were new guidelines for
online course materials; development supported by a central
team. |
| F6 |
One respondent refers to monitoring of new materials and
that this can be problematic due to time/staff factors. Questionnaire
response notes minor or no change to procedures. |
| F7 |
Four respondents note that materials are quality checked
and that new procedures for checking materials have been developed;
posting of materials controlled and restricted to checked materials.. |
| F8 |
No new procedures as existing quality procedures can cope;
all new courses require internal approval. |
| F9 |
No changes except that quality assurance manual available
online. |
| F10 |
One individual referred to considerable changes as fully
quality assured materials could be developed and used by others
within the institution; online materials easily available for
internal and external verification; attempt to implement QA
development cycle of proofing, piloting and then accrediting
materials. |
Figure 8-2: Impact of networked
learning on monitoring development of materials
| |
Changes in quality procedures |
| H1 |
Quality procedures do not appear to have changed yet, although
one respondent spoke about raising awareness of the need to
detail specific quality issues for e-learning, and another
referred to a committee responsible for approving new courses
and, new programmes being offered via the UKeU. The focus is
on quality procedures for computer assisted assessment. |
| H2 |
One respondent noted that the University has adopted the
QAA guidelines on distance learning as policy but they have
not yet put in place specific guidelines for e-learning. Two
other respondents indicated that there had been no real change
for teaching but that some changes for assessment were being
considered. |
| H3 |
One person said that the QA department is working towards
changes. |
| H4 |
Three respondents suggested that minor changes in quality
assurance procedures had occurred. |
| H5 |
No changes except for the issue of plagiarism; plagiarism
is increasing because of the technology. They had a major working
party that looked into plagiarism. |
| H6 |
Limited mention of this issue with a suggestion that there
has been no change to current procedures. |
| H7 |
There was no mention of quality procedures in the interviews |
| H8 |
No changes to main quality procedures (as these are seen
as robust). |
| H9 |
There was limited mention of quality procedures; one respondent
noted the issues about quality of external websites. |
| H10 |
Three respondents mentioned that there had been considerable
changes to quality procedures. |
| F1 |
Three respondents spoke about changes in procedures for putting
materials online (see Figure 8 2 on materials development). |
| F2 |
Two respondents stated that there had been major changes
in quality procedures. |
| F3 |
Three questionnaire responses indicated that some major changes
have occurred in Quality assurance procedures, however, details
of changes not available. |
| F4 |
One respondent noted that they are enhancing their quality
intranet site to ensure that people are using the correct documentation. |
| F5 |
Considerable emphasis by two respondents on changes to quality
assurance due to networked learning; specific sections of annual
quality review document referred to. |
| F6 |
Anticipated changes to quality procedures were mentioned
by several of the respondents but there were no examples of
actual changes having taken place. |
| F7 |
Three respondents mention changes to procedures in terms
of checks of materials before they are posted on the VLE. |
| F8 |
Four respondents mentioned quality issues but stated that
there had been no actual change to current procedures, which
are seen as robust and able to cope. |
| F9 |
There was no mention of changes to quality procedures but
QA manual available online. |
| F10 |
Two respondents stated that internal and external verification
now fully online, perceived as providing consistency. Currently
trying to change quality assurance procedures to encourage
each faculty to undertake own monitoring of materials development.
For e-college feedback to student agreed to be within three
days. |
Figure 8-3: Impact on quality procedures
| CS |
No. of refs. |
Comments |
| H1 |
4 |
Increased transparency of teaching materials and quality
requirements and information will lead to a greater sharing
and better understanding. This will force academics to think
about what the learning outcomes are, to tell students what
they are expected to learn and students now get more information. |
| H2 |
2 |
Greater visibility will allow staff to share good practice
and ideas; along with a greater need to monitor quality of
teaching materials because of this greater visibility |
| H3 |
2 |
Greater visibility of things, previously paper based, that
go on in different schools and an increased need for standardisation;
there is a greater need to ensure constant vigilance and updating
of materials and information. |
| H7 |
1 |
There may be issues in relation to data protection if course
notes are put online |
| H10 |
4 |
Changes to quality procedures, especially in relation to
content as this was now more visible and stated that attention
to content in relation to QA had to some extent been driven
by networked learning; issues in relation to copyright are
influenced by the visibility of the materials if they were
used on the VLE |
| F1 |
3 |
Materials quality has to be of a higher standard and consistent
because of greater visibility; it changed the atmosphere in
the college in terms of quality because of need to be accountable;
there is a greater need to ensure updating of materials and
information. |
| F2 |
2 |
Teaching materials now accessible to all students, some of
which are better than other, and this led to the decision to
only have one agreed set of resources available. There is definitely
a pressure on academic staff because materials are more visible
- less attractive sites may be less successful in recruiting
students. |
| F3 |
2 |
Transparency is having an effect. MIS information is now
more important because it has a knock on effect on attendance
tracking. Information is a lot more visible, a lot more accessible,
in terms of what staff need to do their jobs. |
| F4 |
1 |
Student information is now available online and is much more
visible, this makes life easier because information is readily
available for everyone, but more difficult because you have
to make it available and, in the past, you might not have bothered
to collect all the relevant information. |
| F5 |
3 |
Three respondents noted that higher level of visibility of
admin and teaching led to requirement greater requirement for
monitoring of standards. |
| F6 |
1 |
One respondent commented on the fact that handouts may contain
materials that were copied and that this possibly infringed
copyright legislation. |
| F7 |
4 |
Two respondents noted that the MLE was useful for tracking
student progress, two commented on possible fear amongst the
academics that their materials would be judged as inadequate |
| F10 |
2 |
Tracking noted as a quality issue: the system allowed for
tracking of students and for all to access materials; the visibility
of material would show up poor quality work and possibly deter
some from producing materials. |
Figure 8 4: Other issues (perceptions
of quality)
|
