Grants assessment program is tuned to its audience

Good mixing behind community broadcasting panels

The CBF has a multi-pronged approach to assessing grants, which Ms Boucher said was arguably “the most important task in our operations”.

As far as the people go, the CBF’s peer assessment process has three main parts.

First, an assessor team of more than 130 “virtual” volunteers conduct initial assessments online with the SmartyGrants system. Panels of eight assessors are assigned to each application and given at least three weeks to consider them.

Second, seven-member grants advisory committees review assessments and make recommendations. Lastly, the CBF board gives the final approval.

The CBF aims to ensure assessment times are no longer than three months, with schedules set out a year in advance.

All three bodies ­– the assessor team, the grants advisory committees and the board – have members drawn from community media organisations, although the board also has some independent directors. The committees and the board use a skills and diversity matrix to make appointments that reflect a healthy mix of backgrounds and expertise.

Ms Boucher said the assessor assignment process was “quite complex” and that “the right mix differs from application to application”.

“We match-make applications … to ensure that the people with the know-how are considering the right applications.

“For example, we might have an application from a First Nations radio station that are applying for some salary support, training, studio equipment and new ‘playout’ software.”

She said in that case the CBF would ensure it had First Nations assessors considering the application, with the skills and experience to make informed decisions.

If the organisation uncovers skills gaps in its assessor team, it runs targeted recruitment campaigns, such as for gender-diverse people with knowledge of broadcast transmission technology.

Foundation leans on its long history of volunteer support

Ms Boucher admitted “it can be challenging to fill these gaps” but said budget constraints prevented the organisation from paying volunteers to do assessments.

“Paying them would certainly help if we could afford it – they certainly deserve it – [but] we have to suffice with a lot of ‘thank-yous’, adulation and the odd prize,” she said.

“As the community media sector is built on a strong ethos of volunteerism, that thankfully flows through to our volunteer [assessor] roles to some extent.”

Managing conflicts critical in broadcasting environment

One of the challenges of having sector volunteers assess grants is the risk of conflict.

“Often we have people assessing in the same grant round to which they are applying, so there’s a fair bit to manage in terms of conflict of interest,” Ms Boucher said.

The CBF conducts a “robust process” to ensure that people are not able to assess their own applications.

But Ms Boucher stressed that there were also advantages of having assessors volunteering in the field.

“We find that there is an element of ‘assessing in good faith’ (in that) assessors want to do a good job, because they want to assess the way they would expect their own application to be assessed.”

She said having applicants assess similar grants means they become better grant writers themselves.

“It’s win-win”.

Assessor support crucial for making good decisions

Ms Boucher said good assessors are expected to have a high level of attention to detail, while avoiding getting bogged down in irrelevant information.

“We train our assessors to apply an impact lens to applications without being seduced by impressive grant writing that may not have much substance behind it.”

That training includes online assessment moderation workshops in small groups with sample applications.

Ms Boucher said assessing could be an “intense job” and the CBF has introduced several measures to reduce the burden on volunteers.

This includes encouraging assessors to limit assessments to “a few per day”, as well as setting a maximum number of assessments per round, she said.

And if there are too many applications in the in-box, the CBF calls on additional assessors to ease the load.

In addition, the CBF usually has assessors work on just one of its two grant rounds each year.

“We have enough people in the team that there is a buffer for people to ‘sit out’ a round due to work and life commitments, and we offer refresher training for anyone returning after a break,” Ms Boucher said.

As part of quality control, the CBF reviews all assessments for scoring consistency, bias, and misreading of assessment criteria, and provides additional training and support as needed.

Those reviews are also critical in avoiding incorrect and biased assessments, she said.

Humans remain better than robots, for now

Asked whether artificial intelligence systems could soon be used to assist grants assessors, Ms Boucher responded “never say never”, but stressed that the CBF’s existing processes were working well.

“The human element of assessing the outcomes of our grant applications for community radio and TV stations, and content makers, and their listening and watching communities, will always be at the heart of our process. No robots just yet!”

But she also said the foundation would “always be open to embracing technology that eases the burden on our human assessors”.

She said it employed the various tools available in SmartyGrants for that purpose.

“We’re committed to a process of continuous improvement in terms of keeping up with SmartyGrants developments that can streamline our assessment processes,” she said.

More information

CBF documents: Grants information

Community broadcasting’s power to create social connection

Help sheet: Assessment panels

Best practice: More SmartyGrants guidance on assessments and decisions

SmartyGrants users: Help for assessors

SmartyGrants Grantmaking Toolkit: Great tips for grants assessments