Friday, January 18, 2019

Published 7:06 PM by with 1 comment

Handling Outliers in Linear Regression

What are some of the techniques for handling outliers in linear regression, and how do they compare? I evaluate several in Python.

Methods

I'm taking sample data with a few different types of outliers, and calculating the slope and intercept using the following methods:

For all but LTS, I used scikit-learn. I wrote a custom implementation of LTS because I could not find a nice one when I searched. The LTS algorithm is basically:
  1. randomly sample 60% of the points, perform simple linear regression on them, and repeat 20 times
  2. keep the sample from step 1 that gave you the best score
  3. replace a point in the sample with another point from the original pool of data, perform simple linear regression, and calculate the score; if it improved, keep the newpoint; repeat a bunch of times

Results

I used three outlier types:
  1. 20% of points are all way-off in the same direction
  2. 20% of points have large, random errors added to them
  3. 1 point is massively off; error is 50x the total scale of the data
Overall, simple linear regression resulted in noticeable errors for all three outlier types. All three of the other methods worked well, and LTS and Theil-Sen gave the best results for this specific data set and outlier type.

With an outlier free slope of 1 and intercept of 0, these are the results:

outliers in one direction


ransac regression example

ideal slope: 1 ideal intercept: 0 simple linear regression slope: 0.647 simple linear regression intercept: -1.503 RANSAC slope: 1.03 RANSAC intercept: -2.132 Theil-Sen estimator slope: 0.999 Theil-Sen intercept: -0.004 least trimmed squares slope: 1.0 least trimmed squares intercept: -0.003


random outliers


theil-sen estimator example

ideal slope: 1 ideal intercept: 0 simple linear regression slope: 0.639 simple linear regression intercept: 8.915 RANSAC slope: 0.997 RANSAC intercept: -0.111 Theil-Sen estimator slope: 1.0 Theil-Sen intercept: 0.006 least trimmed squares slope: 1.0 least trimmed squares intercept: -0.004


one big outlier


least trimmed squares regression example

ideal slope: 1 ideal intercept: 0 simple linear regression slope: 0.97 simple linear regression intercept: -48.996 RANSAC slope: 1.0 RANSAC intercept: 0.004 Theil-Sen estimator slope: 1.0 Theil-Sen intercept: -0.001 least trimmed squares slope: 1.0 least trimmed squares intercept: 0.001



      edit

1 comment:

  1. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.

    Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


    Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.

    ReplyDelete