Data set with all the conceivable errors

June 14, 2016 Etc R data

As I was preparing for an R intro course I came up with the idea of creating a fake data set that is stuffed full of all the conceivable errors one can imagine. Just in case my imagination falls short, I’d appreciate all the suggestions in the comments so that I can incorporate more errors.

There is a Hungarian saying about the veterinarian’s horse to describe a case that exhibits all the possible conditions a subject can suffer from (read more of the etymology here). I would like to create a data set that shows all the possible errors a data set can exhibit. This data would be then used in the aforementioned course to make participants’ life miserable experience more diverse.

So far I have been able to come up with the following issues:

  • ill formatted entries, usually as GIS output: "1,234,567.0058654" (needs to clear commas, turn it into numeric, digits are irrelevant but eating up memory)
  • special characters (e.g. from MS Word) where UTF-8 or ASCII is expected
  • mixed case typos: "W-123" vs. "w-123"
  • leading/trailing whitespace: "W-123" vs. "W-123 "
  • MS Excel turning values into dates (e.g. 0-3 works fine, but 3-5 becomes 05-Mar)

I don’t imagine that this list can ever be complete, but right now it is far from complete. If you have struggled with a problem in the past and would like others to learn from it, please leave a comment and I will expand the list accordingly.

Statistical computing meets biodiversity conservation and natural resource management

Phylogeny and species traits predict bird detectability

It all started with this paper in Methods in Ecol. Evol. where we looked at detectability of many species. So we wanted to use life history traits to validate our results. But we had to cut the manuscript, and there was this leftover with some neat patterns, but without much focus. It took a few years, and the most positive peer-review experience ever, and the paper is now early view in Ecography. This post is a quick summary of the goodies stuffed inside the lhreg R package that makes the whole analysis reproducible, and provides some functions for similar PGLMM models.

ABMI (6) ARU (1) C (1) CRAN (1) Hungary (2) JOSM (2) PVA (2) PVAClone (1) QPAD (1) R (18) R packages (1) bioacoustics (1) biodiversity (1) birds (2) course (2) data (1) data cloning (4) dclone (3) dependencies (1) detect (2) detectability (1) footprint (3) forecasting (1) functions (3) intrval (3) lhreg (1) mefa4 (1) monitoring (2) pbapply (5) phylogeny (1) plyr (1) poster (2) processing time (2) progress bar (4) publications (2) report (1) sector effects (1) single visit (1) site (1) slides (2) special (3) species (1) trend (1) tutorials (2) video (4)