The Ethics Questionnaire and its Companion Document

Let us guide you in having a close look at your nascent project. Use the Questionnaire and its Companion Document below to identify the most relevant questions to answer or to pass them on to us!

What is the Ethics Questionnaire?

At some point, someone in the organization may ask you to fill in the Ethics Questionnaire or have a look at it. It lists the most relevant parameters of what may become a CorrelAid project—ethicality issues more narrowly, and privacy or data protection issues more broadly. By filling in the Questionnaire, you provide a structured set of information for people (especially the Ethics Board) to evaluate where your project may need some tweaking or some extra safeguards.

The Questionnaire is here:

https://ee.correlaid.org/x/r5U5hM57

When you fill it in, have a look at the Companion Document, which we present to you below.

The Companion Document

The Companion Document wants to help you to fill the questionnaire in the scoping process by directing your attention to issues that you might want to think about before they actually become issues. You are an ethical person. But the devil is in the details and nobody ever has all of them present at all times. Therefore, this document is a way to help you get an overview, so you don’t miss anything. Once you send off the filled-in questionnaire, we will learn a bunch about your project, and so will you. Also, feel free to talk to the Ethics Committee where it seems this questionnaire raises things you would like to discuss.

Introduction and the Preliminaries section

Who should fill out the questionnaire, and when?

The general idea and the Ethics Committee’s prior assumption about each project is that there is at least one CorrelAid member (from here on: the Project Lead) who is leading - or, if still in ideation phase, scoping the goals and strategy of the project together with one or several people from the Partner organization (including the Contact, often the person who got in touch with CorrelAid). While the Contact knows and communicates what the Partner organization would like from the project, it is the Project Lead's role to know and clarify what is compatible with the values and resources of CorrelAid. So it is rather early—before and during the project scoping phase—that the Project Lead should familiarize themselves with this questionnaire, so the project goals are viable.

The questions concern three main areas: (1) Is the Partner organization eligible to be aided by a pro bono CorrelAid project? (2) Are the project's goals and general strategy in line with CorrelAid guidelines and values? (3) Are the goals and implementation of the project in line with the relevant data protection laws, especially the GDPR? Since you can save the questionnaire as a work in progress, it is a good idea to already fill in the Preliminaries section at the top, to identify the Lead (who is filling this out), the Partner, the project title and to already add descriptions of the Partner organization. If scoping has already advanced, you can also outline the goals and general strategy of the project here.

The Partner and Project

Please give a brief description of what the project will be about

Give an abstract or executive summary of how you would describe the project. Around 250 words should be enough.

What is the cooperation Partner's legal form?

The legal form can already indicate whether we are dealing with a nonprofit. Some organizations are inherently nonprofit (for instance, e.V., gGmbH, gUG, eG), others are not (businesses for instance). It can also become important for several reasons if the Partner is located outside Germany.

When an organization is of the legally nonprofit type, this already means you have less justifying to do; when it is a corporation, a business or other type that does have the right to make a profit, you will have to justify in the comment box how it is still okay for a group of data enthusiasts to invest their time in this project pro bono.

Also, here is where you can elaborate if for some reason the Partner is not an organization, or not a single organization, or why else the assumption of them being an entity is beside the point.

Does the Partner have the funds to pay someone to do this or an equivalent project?

Free qualified work should be provided to those who need it. Potential Partner organizations, even nonprofits, should not be so large and/or well-funded as to make CorrelAid's help into a mere bump in this quarter's bottom line. Examples:

• The Partner organization is a large nonprofit that could provide funding for projects as the planned one.

• The Partner organization actually has funds earmarked for such activities as the one that the project focuses on.

Please verify that the organization in question could not do this project or a similar one on its own. For the comment box: If the Partner does have funding, how is the project justified nevertheless? Why does the Partner not bring up the funding to do this project themselves?

Is the Partner aligned with CorrelAid's values?

The condition is that the Partner organization should not state values or show behaviors that are incompatible with CorrelAid's. You can look those up at our Code of Conduct and Code of Ethics. If you check “Yes”, it means that you have read or are aware of the Code of Conduct and find the Partner overall in alignment.

For the comment box: Comment on any even minor qualifications. No judgment if they later turn out to be irrelevant. This may be the only opportunity for us to discuss alignment, as these things usually do not directly have a bearing on the ins and outs of a project. It is unlikely that any cooperation is off the table just on account of you bringing up nuances that you would like to highlight here.

Also, if the Partner's values do deviate from CorrelAid's, how is the project justified nevertheless?

What positive effects will this project have?

Think not just about the immediate benefit for the Partner organization, but about the greater scope: How does the project help make a positive contribution for the greater social good?

We collected some guiding questions that can help you to think of positive effects:

• What individuals, groups, demographics or organizations will be positively affected by this project? How?

• How are you measuring and communicating positive impact?

We also collected some examples of positive effects from previous CorrelAid projects. For instance: “Thanks to the project,

• … non-data experts can visualize the survey results in an automated and consistent manner.” [read about an example]

• … non-data experts can interact with the data collection tool’s API in a tailored and simplified manner.”

Note that the guiding questions and examples are meant to help you fill the questionnaire but are by no means exhaustive.

Can our volunteers learn something in the project?

CorrelAid also understands itself as having an educational purpose towards its members. Data4Good projects should bring benefits to all sides, Partners and project members. Therefore, it is important to point out what people can learn here. This aspect should not be hard to include and can be as trivial as project members learning how to use a particular tool or library with a particular type of data. For instance:

• Project members learn how to build a dashboard in R/python.

• They learn how to implement sentiment analysis.

• Project members are willing to present their implementation of an analysis pipeline at a local CorrelAid meetup.

Please examine if learning is truly involved in your project; not all activities lend themselves to meaningful learning experiences—for instance mere data entry.

Can the project potentially cause harm?

Harm can come about in many ways and at different points of the project. In this section we collected a range of different forms of harm that data-related projects can cause—not to demotivate or scare you, but to help you exclude any issues you may have been unaware of. In case of any doubt, you can get in touch with us.

Are you aware of any way in which your project could risk breaking the law? Note that we don’t expect you to know all of them or foresee the future but we would like you to share any doubts you do have by the moment you fill the questionnaire. Consider:

• Human rights, data protection, IP and database rights, anti-discrimination laws;

• Data sharing policies;

• Regulation and ethics codes/frameworks that are specific to sectors (e.g., health, employment, taxation).

Here we tried to collect the forms and examples of undesired consequences from an otherwise perfectly successful project. Note that the guiding questions and examples are meant to help you fill the questionnaire but are by no means exhaustive:

• Is there a risk that the collected data or analyses allow re-identification of individuals upon sharing, even after personal data had actually been anonymized?

• Are the data sources that are involved in any way limited so they could influence your project’s outcomes? Consider:

  • Possible bias in data collection, such as inclusion or exclusion of particular groups of people. Could analysis of these data discriminate against any of them?

  • Missing data and data quality: Sometimes data analysis turns out to be deficient to an extent that no analysis may indeed be better than a poor-quality analysis. Is this relevant to your project and are there guard rails to detect such a situation?

  • Data provenance: Do you trust the available data and the way they were collected?

  • When you create a survey to collect data, make sure that values can be entered reliably. For example, make sure the survey form allows a wide age range, short or long names, special characters, postal codes with leading zeros, and so on.

• Issues related to machine learning and AI (skip if this is out of the project’s scope):

  • Is there a risk that a machine learning model could discriminate against any groups? For example, a language model trained on 20th century data might reflect a role for women in society that is no longer acceptable today.

  • Is your model intended to make an automated decision with significant impact for an individual or society? Is it ethical to transfer a decision to an algorithm in this situation?

  • If your model is intended to make predictions for individuals, remember that models usually treat people as groups with similar characteristics and that models make mistakes. People who belong to a group but are exceptions may be treated unfairly by such models.

  • By default, the models learn to fit the data they are trained on as well as possible. However, if the model is intended to make predictions that might have significant consequences for individuals, it might need not only a performance objective but also a fairness objective. For example, if a job advertising model learns that some job historically used to be more likely done by men, it will continue showing such job ads to men, which will exacerbate the existing gender bias.

  • Models that make predictions for people sometimes attempt to imitate a too complex reality based on some assumptions about this reality. If these assumptions are wrong, a model can cause damage. For example, a model that assumes that poor grades of pupils in one class indicate poor teaching can “punish” a teacher, whereas the true causes can be very different.

  • You might want to check whether your model receives “feedback” about its mistakes. E.g., a model that accepts or rejects job applicants may not receive feedback if it never sees whether those who were rejected could in fact be successful and valuable employees.

  • If your model has potential impact for an individual or society, you might want to check that your model is interpretable., i.e., that it is clear how it makes predictions, e.g., which features it uses the most when predicting. For example, simple models like linear regression or decision trees are considered interpretable whereas deep neural networks are considered “black boxes”.

Consider the possibility of the project not working out.

• For instance, if one project member's laptop gets stolen, could sensitive data be affected by this? (Again, since you hopefully read this ahead of finalizing your plans: You could now arrange for this not occurring, e.g. by implementing project-wide encryption, and mention it in your response.)

• If project members fall out with each other, could work turn out unfinished because only one project member has the necessary expertise and all others would be unable to make up for it? Are there safeguards to still land the project gracefully after such a conflict? Keep in mind that the Partner may rely on the project, so that its failure could in fact represent harm to them.

Is the project sustainable for the Partner?

Does the project do more than only fix a symptom where a more systemic approach would be better (even if outside the reach of CorrelAid)? Can instead an outcome be achieved that benefits the Partner beyond the short run, e.g., enables them to solve similar problems autonomously? For instance:

• Instead of preparing a given data analysis, could there be a targeted knowledge transfer in the form of learning materials with the Partner’s data contexts as object of demonstration?

• Will a solution (e.g., a dashboard or a machine learning model) work with new incoming data? With new incoming users?

• Will the Partner be able to identify when a solution becomes outdated and should be updated (e.g., a tool gets deprecated or a machine learning model degrades)?

• Will there be a necessity of knowledge transfer between the project team and the organization and is there someone in the organization that can take over tasks and knowledge from the team?

Will you make the project's outcomes (for instance, scripts, infrastructures, data or analyses) publicly available, so others can benefit from them?

Your project may be solving more people's problems than you think. Consider making public what is elaborated in the project. This includes both communicating at CorrelAid events (local or organization-wide) the team's lessons learned during the project, and making code available. Also, consider the potential extra work that this may take or how it may impact the process: Good shared works contain documentation, commenting, and clean code. Preparing your work for sharing may add workload that you should consider in the scoping process. You may want to keep an eye on documenting interesting insights that may help others avoid issues you had to deal with.

Data Protection and GDPR

This set of questions reflects the rules that the General Data Protection Regulation (GDPR), or Daten·schutz·grund·ver·ordnung (DSGVO), puts in place. Naturally, this is relevant for organizations that deal with data, and especially data about people. For this section, you find guidance right next to the questions themselves in the questionnaire.

Sources

• Webinar “Datenschutz im Ehrenamt: Datenbestände Rechtskonform Nutzen” [link]

• Data Ethics Canvas [link]

• O‘Neil, Cathy (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books

• Wulf, Jessica (2022). Automated Decision-Making Systems and Discrimination Understanding causes, recognizing cases, supporting those affected. Algorithm Watch [link]