/ Recent content on Kun Ren's Blog Posts Hugo -- gohugo.io en-us Sun, 12 Mar 2023 00:00:00 +0000 /2023/03/12/webr-performance/ Sun, 12 Mar 2023 00:00:00 +0000 /2023/03/12/webr-performance/ The webR project is an innovative version of R that has been compiled for the browser and Node.js using WebAssembly, which is a low-level language used for the web. This means that users can run R code directly in their web browser without requiring an R server to execute the code. The annoucement demonstrated the basic functionality of webR including the capability to emulate an R REPL in web browser, execute R code, load R packages, and plot data. /2023/02/21/learning-data-table-helper-functions/ Tue, 21 Feb 2023 00:00:00 +0000 /2023/02/21/learning-data-table-helper-functions/ One of my favorite parts of data.table is that it provides a collection of helper functions that are essential in many use cases. Sometimes I wonder why such basic functions are not provided by base R in the first place. Following are some of the helper functions I find most useful in my every day use. fread(), fwrite() These functions are similar with read.csv() and write.csv() but they are much faster and more memory efficient thanks to the highly optimized implementation in C code, and the use of multi-threading to speed up the process. /2023/02/17/learning-data-table-reference-semantics-and-its-pros-and-cons/ Fri, 17 Feb 2023 00:00:00 +0000 /2023/02/17/learning-data-table-reference-semantics-and-its-pros-and-cons/ In the previous post, I reviewed the basic syntax and design of data.table. In this post, let’s take a closer look at the := operator and its reference semantics. It is one of the key features that sets data.table apart from other data manipulation packages in R. data.table puts performance on its priority from the very beginning. It is designed to be fast and efficient in handling large data sets. /2023/02/08/learning-data-table-the-syntax-and-its-design/ Wed, 08 Feb 2023 00:00:00 +0000 /2023/02/08/learning-data-table-the-syntax-and-its-design/ In my every day work, data.table has been playing a crucial role to work with tabular data for more than 8 years. It has its pros and cons in terms of syntax and extensibility, yet I don’t find a better alternative to it when it comes to handling large in-memory tables with great performance. For beginners, it might be a bit challenging to learn the syntax of data.table in the first place. /2022/03/06/my-recommendations-of-vs-code-extensions-for-r/ Sun, 06 Mar 2022 00:00:00 +0000 /2022/03/06/my-recommendations-of-vs-code-extensions-for-r/ As I have been constantly using R in VS Code for more than two years, the following is a list of my recommendations for VS Code extensions to improve the experience of R development and data research. vscode-R The R extension for VS Code. It provides the R language support and interacting with R terminals. The project wiki is a good place to get started and know more about the features. /2022/03/06/why-i-switched-from-rstudio-to-vs-code/ Sun, 06 Mar 2022 00:00:00 +0000 /2022/03/06/why-i-switched-from-rstudio-to-vs-code/ Like many R users, I had been using R with RStudio since 2013 which was only two years after its initial release. Although RStudio was much simpler than it is today, its simplicity and straightforwardness made it a great, if not the best, choice for most R users I believe. However, when my work gets more and more complicated, sometimes I find it painful to use the free RStudio Server in some scenario. /2020/11/08/using-parse-data-to-analyze-r-code/ Sun, 08 Nov 2020 00:00:00 +0000 /2020/11/08/using-parse-data-to-analyze-r-code/ R’s meta-programming capabilities is one of my favorite aspects of R. We could alter a parsed expression like ordinary objects and evaluate it in a customized environment. This is so-called non-standard evaluation. It makes R so flexible and powerful in data wrangling. However, things could easily mess up if the standard and non-standard scoping are not carefully designed. Instead of manipulating expressions and scoping rules in fancy ways, we could analyze R code and see if there are potential issues before running. /2020/09/13/debugging-r-in-vscode/ Sun, 13 Sep 2020 00:00:00 +0000 /2020/09/13/debugging-r-in-vscode/ R terminal supports interactive debugging via browser() command. When browser() is hit, the R terminal enters the debugging mode so that user could evaluate any R expression in the current environment, e.g. inspect the variables (ls.str()), or run commands such as print the calling stack (where), continue excution (c), go to next statement (n), step into function calls (s), etc. If you want to know more, Advanced R: Debugging is quite helpful on this topic. /2020/06/16/using-httpgd-in-vscode-a-web-based-svg-graphics-device/ Tue, 16 Jun 2020 00:00:00 +0000 /2020/06/16/using-httpgd-in-vscode-a-web-based-svg-graphics-device/ Showing graphics produced by R in external code editors (e.g. VSCode) could be hacky. First, we need to overwrite the default graphics device so that calling plot does not immediately result in a new window we have no control. Instead, we use a pdf(NULL) device that is virtual and can be used to replay user produced graphics in another file-based device (e.g. png(file)) if we enable dislaylist: options( device = function(. /2020/04/14/writing-r-in-vscode-working-with-multiple-r-sessions/ Tue, 14 Apr 2020 00:00:00 +0000 /2020/04/14/writing-r-in-vscode-working-with-multiple-r-sessions/ It could be frustrating when I have to temporarily switch to another project to work with in RStudio Server while the current session is still useful to keep. However, multi-session support is part of RStudio Server Pro but not the free version. In recent months, I have switched my development environment from RStudio Server to VSCode (see my previous posts about languageserver and vscode-R). In VSCode, I can choose to always send commands to active terminal. /2020/03/31/a-simple-way-to-show-stack-trace-on-error-in-r/ Tue, 31 Mar 2020 00:00:00 +0000 /2020/03/31/a-simple-way-to-show-stack-trace-on-error-in-r/ In many programming languages, if an error occurs, the error message typically includes stack trace that helps user indentify the root cause of that error. In R, however, the default error message only contains the last call where error occurs and the error message. This is not very helpful if there are multiple levels of calling stack. The following is a simple example: foo <- function(x, y) { x + y } bar <- function(x, y) { foo(x + y, x - y) } An argument of the incompatible type would make the function call fail. /2019/12/26/writing-r-in-vscode-interacting-with-an-r-session/ Thu, 26 Dec 2019 00:00:00 +0000 /2019/12/26/writing-r-in-vscode-interacting-with-an-r-session/ This article will be updated to reflect the latest vscode-R features. In the previous post, I introduced the features languageserver enables the code editor to provide when we are editing R scripts. But this is only part of the story. A major part of using R is about exploring data interactively, which requires some extent of interactivity between the code editor and a live R session. The latest vscode-R implements an experimental feature called R session watcher, which makes it possible for VSCode to communicate with any live R session under a wide range of scenarios. /2019/12/11/writing-r-in-vscode-a-fresh-start/ Wed, 11 Dec 2019 00:00:00 +0000 /2019/12/11/writing-r-in-vscode-a-fresh-start/ This article will be updated to reflect the latest languageserver features. RStudio has been my first R development environment since I started to work with R five or six years ago. The RStudio user interface is straighforward and easy to work with as it presents source code, R console, environment, files, and plots all in one screen. As my work got complicated, I needed to perform much heavier computation on much larger data of hundreds of gigabytes which only powerful servers could handle. /about/ Tue, 03 Oct 2017 17:15:30 +0800 /about/ I’m a hedge fund researcher, architect, and developer. I specialize in quantitative research and related development of framework and tooling using R and C/C++. I was a former Microsoft MVP (retired in July 2024). I love coding and enjoy participating in the open-source community on GitHub. Following are some major projects I spend time on: As a collaborator: languageserver: An implementation of the Language Server Protocol for R vscode-R: R Extension for Visual Studio Code data. /2014/12/03/tips-on-non-standard-evaluation-in-r/ Wed, 03 Dec 2014 00:00:00 +0000 /2014/12/03/tips-on-non-standard-evaluation-in-r/ One of my favorite features of R is its meta-programming facilities. It can be simply demonstrated by the following examples. An ordinary use of R is to do statistical computing. We can evaluate something like sin(0) [1] 0 Meta-programming in R allows users to manipulate the expression to evaluate. We can use quote to create an object that represents a function call. quote(sin(0)) sin(0) In this way, sin(0) is not evaluated but parsed as a call object which basically can be represented as a list of function name and the arguments. /2014/09/22/piper-tutorial-is-released/ Mon, 22 Sep 2014 00:00:00 +0000 /2014/09/22/piper-tutorial-is-released/ Twelve days after the initial commit, pipeR tutorial is released! If you want to write R code fluently and process data elegantly, I strongly recommend that you read this tutorial which is designed to serve as a complete guide to pipeR package, including how it works with dplyr, rlist, and rvest with vivid examples. You can also provide suggestions at the issue page or fork and contribute. Now I have started working on rlist-tutorial which is yet far from complete. /2014/08/23/more-operators-or-more-syntax/ Sat, 23 Aug 2014 00:00:00 +0000 /2014/08/23/more-operators-or-more-syntax/ The motivation of pipeline operator is to make code more readable. In many cases, it indeed better organizes code so that the logic is presented in human-readable fluent style. In other cases, however, such operators can make things worse. Recently, I had an interesting discussion on how to add side effect piping to pipeR functionality (in this issue), just like the tee operator (%T>%) does. Linlin Yan initially suggested to introduce a new operator to handle this while my opinion is to introduce a new syntax. /2014/08/16/easier-way-to-chain-commands-using-pipe-function/ Sat, 16 Aug 2014 00:00:00 +0000 /2014/08/16/easier-way-to-chain-commands-using-pipe-function/ In pipeR 0.4 version, one of the new features is Pipe() function. The function basically creates a Pipe object that allows command chaining with $, and thus makes it easier to perform operations in pipeline without any external operator. In this post, I will introduce how to use this function and some basic knowledge about how it works. But before that, I would like to make clear that you don’t have to learn a whole new thing if you are familiar with magrittr’s %>% operator or pipeR’s %>>% operator. /2014/08/08/difference-between-magrittr-and-piper/ Fri, 08 Aug 2014 00:00:00 +0000 /2014/08/08/difference-between-magrittr-and-piper/ Pipeline is receiving increasing attention in R community these days. It is hard to tell when it begins but more people start to use it since the easy-and-fast dplyr package imports the magic operator %>% from magrittr, the pioneer package of pipeline operators for R. The two packages co-work well: dplyr works with data frames by a set of basic operations, and magrittr’s %>% operator chains the commands together and makes data manipulation process more readable and consistence with our intuition. /2014/08/07/a-quiz-on-magrittr-how-many-scores-can-you-get/ Thu, 07 Aug 2014 00:00:00 +0000 /2014/08/07/a-quiz-on-magrittr-how-many-scores-can-you-get/ Here is a quiz on magrittr and you may check if you are really good at using it. Since the CRAN version currently does not support nested ., it won’t be interesting to make a quiz on that version. All the following examples are using the latest development version on GitHub. You can do the same test with the CRAN version if you like. Consider the following function: f <- function(x,y = "nothing") { cat("x = ", x, "\n") cat("y = ", y, "\n") } Now answer what is going to be evaluated for the following expressions: /2014/08/06/introducing-rlist-0-3/ Wed, 06 Aug 2014 00:00:00 +0000 /2014/08/06/introducing-rlist-0-3/ rlist 0.3 is released! This package now provides a wide range of functions for dealing with list objects. It can be especially useful when they are used to store non-tabular data. Two notable features are added in this version. First, list.search and equal() are added in support of fuzzy filtering and searching. Second, List object is added to provide object-based, light-weight chaining operation for list objects. In the examples, I will use both rlist and pipeR package for easier coding. /2014/08/04/introducing-piper-0-4/ Mon, 04 Aug 2014 00:00:00 +0000 /2014/08/04/introducing-piper-0-4/ pipeR 0.4 is released! Check it out at the project page. In this new version, two things happen. First, %>>% handles everything. Second, the introduction of Pipe object. %>>% In version 0.4, I decide to merge the functionality of all three operators in the previous versions. Only %>>% operator remains, and the two other operators are deprecated. But to make the transition smoother, the deprecated operators still work but will send warnings. /2014/07/29/scraping-information-of-cran-packages/ Tue, 29 Jul 2014 00:00:00 +0000 /2014/07/29/scraping-information-of-cran-packages/ In my previous post, I demonstrated how we can scrape online data using existing packages. In this post, I will take it a bit further: I will scrape more information of CRAN packages since each of them also has a web page like this. More specifically, I want to build my own database that includes the date, name, title, version, imports, and suggests of the packages. Although there are various ways to get the information of CRAN packages without scraping, this post is intended for showing how a typical scraping works. /2014/07/25/what-are-the-most-popular-keywords-of-cran-packages/ Fri, 25 Jul 2014 00:00:00 +0000 /2014/07/25/what-are-the-most-popular-keywords-of-cran-packages/ A large proportion of R’s power should be attributed to the enormous amount of extension packages. Many packages are published to CRAN. These packages cover a wide range of fields. In this post, I’ll show you how to use R to scrap the titles of all CRAN packages from the web page and find out which keywords are the most popular. To minimize the efforts, we try best to avoid reinventing the wheels and get some answer as quickly as possible. /2014/06/26/rlist-a-new-package-for-working-with-list-objects-in-r/ Thu, 26 Jun 2014 00:00:00 +0000 /2014/06/26/rlist-a-new-package-for-working-with-list-objects-in-r/ In recent years, non-relational data have attracted more and more attentions. Roughly speaking, all datasets that are hard to put into a rectangular table with rows and columns are non-relational datasets. The following data is a very simple non-relational dataset in JSON format. The dataset contains some information of three programmers, each of whom has a name, an age, some interests, and a list of programming languages with the number of years used. /2014/04/15/fit-an-ornstein-uhlenbeck-process-with-discrete-time-series-data/ Tue, 15 Apr 2014 00:00:00 +0000 /2014/04/15/fit-an-ornstein-uhlenbeck-process-with-discrete-time-series-data/ As we know, a Brownian motion is usually formulated as $$dx_t = \mu,dt+\sigma,dW_t$$ which is the continuous case of a random walk. In some cases, it is quite convenient to use this formulation to describe the characteristic of asset prices due to its highly unpredictable behavior. However, there are financial indicators or variables that also exhibit, at least temporarily, stable behavior. For example, two companies with great homogeneity may be reflected in the synchronous co-movement of their stock prices. /2014/04/08/use-pipeline-operators-in-r/ Tue, 08 Apr 2014 00:00:00 +0000 /2014/04/08/use-pipeline-operators-in-r/ In data-driven statistical computing and data analysis, applying a chain of commands step by step is a common situation. However, it is neither straightforward nor flexible to write a group of deeply nested functions. It is because the function that comes later must be written first. Consider the following example in which we need to take the following steps: Generate 10000 random numbers from normal distribution with mean 10 and standard deviation 1 Take a sample of 100 without replacement from these numbers Take log of the sample Take difference of the log numbers Finally, plot these log differences as red line segments These steps only require basic functions. /2014/03/15/a-brief-introduction-to-higher-order-functions-in-r/ Sat, 15 Mar 2014 00:00:00 +0000 /2014/03/15/a-brief-introduction-to-higher-order-functions-in-r/ In R, function may not be as special as it is in other programming languages; it is regarded as one of the many types and can be passed as an argument to some other function. The way we deal with other objects such list and data.frame definitely applies to function. Here is a simple example in which we define two functions and pass them to another function as an argument, respectively. /2014/03/14/reshape-r-data-frame-from-long-to-wide-format/ Fri, 14 Mar 2014 00:00:00 +0000 /2014/03/14/reshape-r-data-frame-from-long-to-wide-format/ Oftentimes, we obtain a long or a wide table from a certain data source, and it may be the only format we can get. For example, some financial databases provide daily tick data for all stocks in a financial market. The data table may be arranged in a long format like this: Code Date Open High Low Close 1 A01 2014-03-10 10.0 13.0 9.0 11.0 2 A01 2014-03-11 11.0 12.0 10. /2014/02/20/extract-information-from-texts-with-regular-expressions-in-r/ Thu, 20 Feb 2014 00:00:00 +0000 /2014/02/20/extract-information-from-texts-with-regular-expressions-in-r/ People love dealing with well-structured data. It costs much less efforts than working with disorganized raw texts. In economic and financial research, we typically download data from open-access websites or authentication-required databases. These sources may provide data in multiple formats. For example, almost all databases are able to provide data in CSV format, which is a widely supported standard format. In R, it is convenient to call read.csv() to import the CSV file as a data frame with the right header and data types. /2014/02/15/a-principle-of-writing-robust-r-program/ Sat, 15 Feb 2014 00:00:00 +0000 /2014/02/15/a-principle-of-writing-robust-r-program/ Writing R code can be very easy. It depends on how much you want to achieve with your code and what features you want your code to support. To test a random thought that needs some statistical evidence, you only need to casually import data, slightly transform the data to a necessary form, and perform some statistical tests and see the conclusions. You don’t need to create a project, model the project structure, design the features, and implement the code from one stage to another. /2014/02/07/use-sql-to-operate-r-data-frames/ Fri, 07 Feb 2014 00:00:00 +0000 /2014/02/07/use-sql-to-operate-r-data-frames/ In both research and application, we need to manipulate data frames by selecting desired columns, filtering records, transforming and aggregating data. R provides built-in functions for data frame manipulation. Suppose df is the data frame we are dealing with. We use df[1:100,] to select the first 100 rows, df[,c("price","volume")] to select price and volume columns, df[df$price >= mean(df$price),] to single out records with prices no less than their average, transform(df, totalValue=price*volume) to add a new column totalValue for each record, apply(df,2,mean) to calculate the mean of each column. /2014/02/01/introduction-to-parallel-computing-in-r/ Sat, 01 Feb 2014 00:00:00 +0000 /2014/02/01/introduction-to-parallel-computing-in-r/ For R beginners, for loop is an elementary flow-control device that simplifies repeatedly calling functions with different parameters. A possible block of code is like this: run <- function(i) { return((i+1)/(i^2+1)) } for(i in 1:100) { run(i) } In this code, we first define a function that calculates something, and then run the function from i = 1 to i = 100. This can be altered to a Monte Carlo simulation in which we estimate the distribution of a statistic or to calculate the theoretical price of an European call option in a binomial tree. /2014/01/28/difference-between-assignment-operators-in-r/ Tue, 28 Jan 2014 00:00:00 +0000 /2014/01/28/difference-between-assignment-operators-in-r/ For R beginners, the first operator they use is probably the assignment operator <-. Google’s R Style Guide suggests the usage of <- rather than = even though the equal sign is also allowed in R to do exactly the same thing when we assign a value to a variable. However, you might feel inconvenient because you need to type two characters to represent one symbol, which is different from many other programming languages. /2014/01/26/r-essentials/ Sun, 26 Jan 2014 00:00:00 +0000 /2014/01/26/r-essentials/ It is quite easy to get started with R. The very first step is to download R from the official website , and install it. I suggest that you install both 32-bit and 64-bit versions for greater compatibility if you are running a 64-bit operating system. For typical statistical programming, if your dataset is not huge, it does not matter which one you run; if you try to process a giant dataset (e. /2014/01/24/r-getting-started/ Fri, 24 Jan 2014 00:00:00 +0000 /2014/01/24/r-getting-started/ R rocks in both academia and industry nowadays. A rapidly increasing number of researchers choose R to be one of their productive tools for data analysis and data visualization. It is partially because the software is totally free and open-source but also because the community behind the stage who contributes to nearly 5000 packages remains growing, which results in an evolving and sustainable ecosystem. If you are a student learning statistics or econometrics, you must have heard about R. /2014/01/23/welcome-to-my-homepage/ Thu, 23 Jan 2014 00:00:00 +0000 /2014/01/23/welcome-to-my-homepage/ Finally I set up my blog website hosted by GitHub Pages. It is quite amazing to create a website totally for free. After some trial and error, now I come up with a stable workflow to publish my work, introductory materials, and random thoughts online. The contents of my postings will be related to data science, statistical programming, quantitative finance, and relevant topics. I will try to strike a balance between theory and application.