readr

Read tabular data tibbles

Comma Delimited files

write_file(x = "a,b,c\n1,2,3\n4,5,NA", path = "file.csv")
read_csv("file.csv")

Semi-colon Delimited Files

write_file(x = "a;b;c\n1;2;3\n4;5;NA", path = "file2.csv") 
read_csv2("file2.csv")

Files with Any Delimiter

write_file(x = "a|b|c\n1|2|3\n4|5|NA", path = "file.txt") 
ead_delim("file.txt", delim = "|")

Fixed Width Files

write_file(x = "a b c\n1 2 3\n4 5 NA", path = "file.fwf") 
read_fwf("file.fwf", col_positions = c(1, 3, 5))

Tab Delimited Files

write_file(x = "a\tb\tc\n1\t2\t3\n4\t5\tNA", path = "file.tsv")
read_tsv("file.tsv") Also read_table()

Useful arguments

Example File

write_file("a,b,c\n1,2,3\n4,5,NA","file.csv")
f <- "file.csv" 

No header

read_csv(f, col_names = FALSE)

Provide header

read_csv(f, col_names = c("x", "y", "z"))

Skip lines

read_csv(f, skip = 1) 

Read in a subset

read_csv(f, n_max = 1) 

Missing Values

read_csv(f, na = c("1", "."))

Read Non-Tabular Data

Read a file into a single string

read_file(file, locale = default_locale())

Read each line into its own string

read_lines(file, skip = 0, n_max = -1L, na = character(),
locale = default_locale(), progress = interactive())

Read each line into its own string

read_lines(file, skip = 0, n_max = -1L, na = character(),
locale = default_locale(), progress = interactive())

Read a file into a raw vector

read_file_raw(file)

Read each line into a raw vector

read_lines_raw(file, skip = 0, n_max = -1L,
progress = interactive())

Read Apache style log files

read_log(file, col_names = FALSE, col_types = NULL, skip = 0, n_max = -1, progress = interactive())

Tibbles

An enhanced data frame

The tibble package provides a new S3 class for storing tabular data, the tibble. Tibbles inherit the data frame class, but improve three behaviors:

• Subsetting - [ always returns a new tibble, [[ and $ always return a vector.
• No partial matching - You must use full column names when subsetting
• Display - When you print a tibble, R provides a concise view of the data that fits on one screen
• Control the default appearance with options:

options(tibble.print_max = n,
tibble.print_min = m, tibble.width = Inf)

• View full data set with View() or glimpse()
• Revert to data frame with as.data.frame()

CONSTRUCT A TIBBLE IN TWO WAYS

• tibble(…) - Construct by columns.

tibble(x = 1:3, y = c("a", "b", "c"))

• tribble(…) - Construct by rows.

tribble( ~x, ~y,
 1, "a",
 2, "b",
 3, "c")

Other functions

• Convert data frame to tibble.

  as_tibble(x, …)  

• Convert named vector to a tibble

  enframe(x, name = "name", value = "value")

• Test whether x is a tibble.

  is_tibble(x) 

data.frame

• no external package needed
• straightforward for simple tasks
• strings are stored as factors per default
• data recycling
• row names can be provided

mydf = data.frame(
    a = c("Paul", "Kim","Nora","Sue","Paul","Kim"),
    b = c("A", "B","C","B","B","C"),
    c = rnorm(2)
)
mydf

sapply(mydf, class)

##         a         b         c 
##  "factor"  "factor" "numeric"

data.table

• quick and easy to code
• start processing time
• great documentation
• second class ‘data.frame’

• ‘data.table’ struction is similar to SQL structure

  • ‘nameDT [i, j, by]’
    • ‘i’ stands for the subset from our ‘data.table’ we want to work with
    • ‘j’ is the actual calculation that will be performed in the data subset ‘i’
    • the whole calculation is grouped by ’by
• strings are not automatically transfromed to factors
• no custom row names, just row IDs

• if a dataset is too big to be processed, then only the first and last five rows are printed

library(data.table)
mytable = data.frame(
    a = c("Paul", "Kim","Nora","Sue","Paul","Kim"),
    b = c("A", "B","C","B","B","C"),
    c = rnorm(2)
)
mytable

data_frame

• requires equals column length
• only columns of length 1 will be recycled
• only the firs couple of rows are displayed in case of large datasets
• second class: ‘data.frame’
• stirngs are not automatically transformed to factors
• no custom row names, just row IDs

library(dplyr)
my_df = data.frame(
    a = c("Paul", "Kim","Nora","Sue","Paul","Kim"),
    b = c("A", "B","C","B","B","C"),
    c = rnorm(6)
)
my_df

Summary

class(mydf); class(mytable); class(my_df)

## [1] "data.frame"

## [1] "data.frame"

## [1] "data.frame"

• it is benefically to use advanced tools for extended data management
• ‘data.table’ and ‘data_frame’ have the standard ‘data.frame’ as second class

nycflights13

To explore the basic data manipulation verbs of dplyr, we’ll use nycflights13::flights. This data frame contains all 336,776 flights that departed from New York City in 2013. The data comes from the US Bureau of Transportation Statistics, and is documented in ?flights.

library(nycflights13)
flights

You might notice that this data frame prints a little differently from other data frames you might have used in the past: it only shows the first few rows and all the columns that fit on one screen. (To see the whole dataset, you can run View(flights) which will open the dataset in the RStudio viewer). It prints differently because it’s a tibble. Tibbles are data frames, but slightly tweaked to work better in the tidyverse. For now, you don’t need to worry about the differences; we’ll come back to tibbles in more detail in wrangle.

You might also have noticed the row of three (or four) letter abbreviations under the column names. These describe the type of each variable:

• int stands for integers.
• dbl stands for doubles, or real numbers.
• chr stands for character vectors, or strings.
• dttm stands for date-times (a date + a time).
• lgl stands for logical, vectors that contain only TRUE or FALSE.
• fctr stands for factors, which R uses to represent categorical variables with fixed possible values.
• date stands for dates.

Base R

Compared to the corresponding base functions, readr functions:

• Use a consistent naming scheme for the parameters (e.g. col_names and col_types not header and colClasses).

• Are much faster (up to 10x).

• Leave strings as is by default, and automatically parse common date/time formats.

• Have a helpful progress bar if loading is going to take a while.

• All functions work exactly the same way regardless of the current locale. To override the US-centric defaults, use locale().

data.table and fread()

data.table has a function similar to read_csv() called fread. Compared to fread, readr functions:

• Are slower (currently ~1.2-2x slower. If you want absolutely the best performance, use data.table::fread().

• Use a slightly more sophisticated parser, recognising both doubled ("""") and backslash escapes ("\""), and can produce factors and date/times directly.

• Forces you to supply all parameters, where fread() saves you work by automatically guessing the delimiter, whether or not the file has a header, and how many lines to skip.

• Are built on a different underlying infrastructure. Readr functions are designed to be quite general, which makes it easier to add support for new rectangular data formats. fread() is designed to be as fast as possible.