class: center, middle, inverse, title-slide .title[ # Consolidating Analysis and Dissemination of Research using GitHub ] .subtitle[ ## Using GitHub for Open-Source Analytics, Reporting, and Dissemination of Research ] .author[ ### Nathan Dadey ] .institute[ ### Center for Assessment ] .date[ ### April 23rd, 2022<br>NCME 2022 Annual Meeting<br>Demonstration Session <a href="https://github.com/CenterForAssessment/NCME_2022_Demonstration/blob/main/Consolidating_Analysis_and_Dissemination_of_Research_Using_GitHub.Rmd" target="_blank" class="github-corner" aria-label="View source on Github"><svg viewBox="0 0 250 250" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path class="octo-arm" d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;"></path><path class="octo-body" d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor"></path></svg></a> ] --- layout: true <!--this adds the footer to all slides, depends on footer class in css--> <div class="footer" style="left: 5%;"><img src="https://mirrors.creativecommons.org/presskit/icons/cc.svg" style = "max-width: 40%; opacity: 0.5;"></img></div> <div class="footer" style="left: 7.5%;"><img src="https://mirrors.creativecommons.org/presskit/icons/by.svg" style = "max-width: 40%; opacity: 0.5;"></img></div> <div class="footer" style="left: 40%;"><a class="footer" href="http://www.nciea.org"> www.nciea.org</a></div> --- class: inverse, center, middle # Consolidating Analysis and Dissemination of Research using GitHub --- # Motivation: Data Driven Documents + In many cases, work can be shared well via one or more **short vignettes or other write ups** (e.g., Allie's excellent vignettes on [multiple imputation](https://centerforassessment.github.io/Internship_2021/articles/PSW_Vignette_Landing_Page.html) and [propensity score weighting](https://centerforassessment.github.io/Internship_2021/articles/MI_Vignette_Landing_Page.html)) -- + Such write-ups can also be thought of as one form of **data-driven documents** (e.g., [NYT](https:/www.nytimes.com/interactive/2017/12/05/upshot/a-better-way-to-compare-public-schools.html) [Articles](https:/www.nytimes.com/interactive/2017/12/05/upshot/a-better-way-to-compare-public-schools.html), which typically: + Are well structured + Provide a limited n umber of comparisons + Meant to tell a predetermined story -- + This approach has been implemented in the in the prior presentations. --- # Motivation: Dashboards and Data Explorers + In other cases, we want to encourage users to **interact with data**, either through dashboards (e.g., [CO](https://edx.cde.state.co.us/SchoolView/DataCenter/reports.jspx), [CA](https://www.cde.ca.gov/ta/ac/cm/), [NYC](https://tools.nycenet.edu/), [MI](https://www.mischooldata.org/DistrictSchoolProfiles2/ReportCard/EducationDashboard4.aspx)) or more elaborate data exploration tools (e.g., [NAEP](https://www.nationsreportcard.gov/ndecore/landing)) -- + These kinds of tools typically: + Are flexible + Support numerous comparisons + Are DIY, allowing users to find their own stories -- + Implementing this approach may involve [flexdashboard](https://rmarkdown.rstudio.com/flexdashboard/) with interactivity supported through packages like [DT](https://rstudio.github.io/DT/), [polt.ly](https://plot.ly/r/) , [htmlwidgets](https://www.htmlwidgets.org/) or Shiny (e.g., see this [simple example](https://www.lexjansen.com/pharmasug-cn/2019/DV/Pharmasug-China-2019-DV19.pdf)). --- # Motivation: A Third Use Case + In this final case, the **full complexities of multiple analysis** may need to be captured (e.g., a larger set of connected write ups): + Well structured across numerous write ups + Contain multiple comparisons supported through multiple analyses + Meant to provide a balance between structured and DIY storytelling -- + One way to implement this approach is by binding up the multiple write ups from multiple analyses (captured in individual . rmds ) into a larger book via the [bookdown](https://bookdown.org/yihui/rmarkdown/books.html) package. --- <img src="Consolidating_Analysis_and_Dissemination_of_Research_Using_GitHub_files/figure-html/pages_to_book-1.png" width="1508" style="display: block; margin: auto;" /> --- <br> <img src="Consolidating_Analysis_and_Dissemination_of_Research_Using_GitHub_files/figure-html/example_workbook-1.png" width="2456" style="display: block; margin: auto;" /> --- # Implementation + Book can be hosted via GitHub Pages by following the approach outlined in section 6.3 of Authoring Books and Technical Documents with R Markdown [GitHub Pages](https://bookdown.org/yihui/bookdown/github.html) [ 6.3 of Authoring Books and Technical Documents with R Markdown](https://pages.github.com/) --- # Considerations This approach may be particularly useful: + For analyses that are **iteratively updated** and **shared** (e.g., a workbook for state data analysis; annual psychometric analyses) -- + As a space to share final analyses, after all the messy, iterative work has been done --- layout: true class: inverse, bottom --- # Supplemental Materials --- layout: false # Some Orienting References + Educational Data Mining ([Romero & Ventura, 2010](https://ieeexplore.ieee.org/document/5524021); [Baker & Yacef, 2009; Pena-Ayala, 2014](https://jedm.educationaldatamining.org/index.php/JEDM/article/view/8); [Dutt, Ismail, & Herawan , 2017](https://ieeexplore.ieee.org/abstract/document/7820050)) + Learning Analytics ([Ferguson, 2012](http://oro.open.ac.uk/36374/1/IJTEL40501_Ferguson%20Jan%202013.pdf); [Verbert et al., 2013](https://journals.sagepub.com/doi/abs/10.1177/0002764213479363)) + Data-Driven Documents ([Bostock, Ogievetsky , & Heer , 2011](https://ieeexplore.ieee.org/abstract/document/6064996)) + Data-Driven Decision Making ([Marsh, Pane, & Hamilton, 2006](https://www.rand.org/pubs/occasional_papers/OP170.html)) + Data Visualization + Classics ([Cleveland , 1994](https://www.amazon.com/Elements-Graphing-Data-William-Cleveland/dp/0963488414); [Tufte , 2001](https://www.edwardtufte.com/tufte/books_vdqi)) + Modern ([Few, 2013](http://www.stephen-few.com/idd.php); [Nussbaumer-Knaflic , 2015](http://www.storytellingwithdata.com/book)) + State Report Cards ([Curl & Peltzman , 2017](http://www.ccsso.org/sites/default/files/2017-11/CCSSO%20Reporting%20Best%20Practice%20Resource.pdf))