How to Use R for Data Cleaning – College Aftermath

R can help you turn inconsistent, incomplete data into a reliable dataset that is easier to analyze. A careful cleaning workflow preserves the original file, makes changes explicit, and checks that each change did what you intended.

Before you begin, decide what one row should represent and which values count as missing or invalid for your project. Cleaning is not simply deleting unusual entries: it means finding problems, applying defensible rules, and recording the results so someone else can understand them.

What You’ll Need

R installed on your computer, plus an editor or development environment where you can run R code.
A copy of the source data in a format R can read, such as a comma-separated values file.
A clear description of the columns, expected types, units, valid ranges, and meaning of missing values. If the data came from a survey or database, consult its documentation when available.
A separate folder or filename for cleaned output, so you do not overwrite the original data.
Optional packages: readr for importing delimited files and dplyr and tidyr for common data-wrangling operations. Install a package once with install.packages("readr"), for example, then load it in a session with library(readr).

Step-by-Step

Keep the source file unchanged. Store a working copy in a project folder and write down where the original came from. In R, use a clear project-specific path rather than relying on whichever folder happens to be active. Check that R is looking in the expected place with getwd(). You can set a working directory with setwd(), but using a project-based setup or explicit relative paths usually makes scripts easier to rerun on another computer.
Import the data and inspect it before editing. For a CSV file, base R can read it with read.csv("data.csv", stringsAsFactors = FALSE). With readr, use read_csv("data.csv"). Save the result under a descriptive name, such as raw_data, and make a separate working object: clean_data <- raw_data. Then check dim(raw_data), names(raw_data), str(raw_data), and head(raw_data). These reveal the number of rows and columns, column names, inferred types, and sample values. Do not assume import inferred every type correctly.
Review column names and data types. Names containing spaces or inconsistent capitalization can make code harder to read. Rename them deliberately, preserving their meaning. For example, names(clean_data)[names(clean_data) == "Student ID"] <- "student_id" renames one column. Check whether dates, numbers, and categories were imported as intended. A numeric-looking column may contain text such as “unknown,” while an identifier made of digits may need to remain text so leading zeroes are not lost. Convert types only after checking what the values represent.
Check missing values and placeholders. R represents a missing value as NA, but a file may use blank cells, "N/A", "unknown", or a special code. Inspect a column with unique(clean_data$status) or count values with table(clean_data$status, useNA = "ifany"). Recode a placeholder only when documentation or a responsible data owner confirms that it means missing. For example, clean_data$status[clean_data$status == "unknown"] <- NA is appropriate only if “unknown” is not a meaningful status. Avoid replacing every unusual value automatically.
Standardize text carefully. Extra spaces and inconsistent capitalization can create duplicate categories, such as “North” and “ north ”. Use trimws() to remove leading and trailing whitespace: clean_data$region <- trimws(clean_data$region). If capitalization is not meaningful, standardize it with tolower() or toupper(). Then inspect the distinct values again. Do not apply text transformations to names or free-text responses if capitalization, punctuation, or wording carries meaning.
Validate numbers, dates, and ranges. Review summaries with summary(clean_data) and inspect suspicious values directly. A column intended to contain ages, scores, or dates may have impossible entries, but the valid range depends on the dataset and its definitions. Convert a column with as.numeric() or parse dates with an appropriate date function only after checking its format. Conversion can introduce new NA values when text cannot be interpreted, so compare missing-value counts before and after. Flag questionable rows for review rather than deleting them simply because they look unusual.
Look for duplicate records. First decide what makes a row a duplicate. Entirely identical rows can be checked with duplicated(clean_data). To check repeated identifiers, use duplicated(clean_data$student_id) or group by the fields that should uniquely identify a record. Repeated identifiers are not always errors: one person may have several observations. If duplicates should be removed, inspect them first and retain a record of the rule, for example by comparing the relevant rows before using unique() or a targeted distinct() operation.
Check relationships and consistency across columns. Look for contradictions that a single-column summary will miss. A start date should not come after an end date, and a total may need to equal the sum of its component fields. The correct rule depends on how the data were collected, so consult the data dictionary or collection process. In R, create a logical check such as clean_data$start_date > clean_data$end_date, then inspect rows where the result is TRUE. Keep a list of issues and decisions rather than silently changing values.
Apply changes in a reproducible script. Put import, cleaning, and validation steps in an R script in the order they should run. Prefer explicit operations to manual edits in a spreadsheet, which can be difficult to reproduce. With dplyr, for example, clean_data <- clean_data |> dplyr::mutate(region = trimws(region)) makes a transformation visible in code. Add comments explaining choices that are not obvious, especially rules for missing values, exclusions, and outliers. Rerun the script from the original file to check that it works from start to finish.
Recheck and export the cleaned data. Repeat key checks: dimensions, column names, types, missing-value counts, distinct categories, duplicates, and validation rules. Compare the cleaned data with the raw copy to make sure only intended changes occurred. Save the result to a new file, for example with write.csv(clean_data, "clean_data.csv", row.names = FALSE). Keep the script and, where appropriate, a short record of decisions alongside the output. Do not label a dataset “clean” merely because the code ran without an error; confirm that its contents meet the requirements for the intended analysis.

Common Mistakes

Overwriting the original: A mistaken recode can be hard to reverse. Keep a raw object and write cleaned output to a new file.
Deleting every row with a missing value: Missingness may affect only a few columns, and removing whole rows can discard useful information. Decide how to handle missing data for each analysis and document the choice.
Assuming unusual means incorrect: A rare value can be valid. Check source definitions and investigate before correcting or excluding it.
Converting identifiers to numbers: IDs are labels, not quantities. Numeric conversion can remove leading zeroes or invite meaningless arithmetic.
Removing duplicates without defining them: Identical rows, repeated IDs, and repeated measurements are different situations. Specify which fields define a unique record.
Making undocumented edits: Manual changes and vague rules make results difficult to reproduce. Put transformations in code and explain important decisions.

Tips

Start with a short cleaning plan: identify the unit of observation, expected types, missing-value conventions, and checks that should pass.
Use descriptive object names, such as raw_data, clean_data, and invalid_dates, so the purpose of each object is clear.
Inspect before and after each consequential change. For a recode, compare category counts; for a type conversion, check how many values became missing.
Keep an audit trail of exclusions and corrections, including the rule and the number of affected rows. When practical, create a separate table of flagged records rather than dropping them immediately.
Use functions and packages you understand. Base R is enough for many tasks; packages can make repeated transformations clearer, but they do not decide what counts as valid data.
Make your script portable by using relative paths and recording required packages. Test it from a fresh R session so it does not depend on objects left over from earlier work.

Troubleshooting

R says it cannot find the file. Check getwd(), confirm the filename and capitalization, and verify that the file is in the folder your path refers to. A relative path is resolved from the working directory, not necessarily from the folder you expect.

A column has the wrong type or unexpected missing values. Inspect its raw entries with unique() or a frequency table. Text mixed into a numeric column can prevent correct parsing. Identify what the nonnumeric values mean before converting, and compare the count of missing values before and after the conversion.

A transformation returns an error about a missing or unknown column. Print names(clean_data) and check spelling, capitalization, and spaces in the imported column name. Confirm that earlier code did not rename the column or create a different object.

The number of rows changes unexpectedly. Check each filtering, joining, and duplicate-removal step separately. Print dimensions before and after it, then inspect the rows affected. A join can multiply rows when its matching key is not unique, so check key uniqueness on both inputs before joining.

The script works only in the current session. Restart R and run the script from the beginning. Load required packages explicitly, create objects in the script, and correct paths. This exposes hidden dependencies on manually created objects or prior console commands.

Frequently Asked Questions

Can I clean data using base R alone?
Yes. Base R can import common files, inspect columns, recode values, find duplicates, filter rows, and export results. Packages are optional and can make some workflows more readable.

Should I remove rows with missing values?
Not automatically. First determine what is missing, how much it affects the fields used in your analysis, and whether the missingness has a known meaning. Choose a method that fits the project and report the decision.

How do I know whether a value is an outlier?
A statistical rule can flag unusual values, but it cannot establish that they are errors. Check units, source records, and domain definitions. Preserve valid extremes and document any corrections or exclusions.

What is the difference between cleaning and transforming data?
Cleaning addresses problems such as inconsistent labels, incorrect types, and invalid records. Transformation changes data into a form suited to analysis, such as creating a new variable or reshaping a table. A step can do both, so describe its purpose clearly.

How should I handle a value that cannot be verified?
Do not guess. Keep the original value, flag it for review, and record what is unknown. If a decision is necessary for analysis, document the assumption and consider whether results change under a reasonable alternative.

How can I make the work easy to repeat?
Save the cleaning steps in a script, start from an unchanged source file, use clear paths, and rerun the script in a fresh session. Keep notes on rules and checks, then verify the exported file before sharing or analyzing it.