scikit-learn Archives - Page 4 of 5

Feature/Variable importance after a PCA analysis

August 14, 2022 by Magenaut

I have performed a PCA analysis over my original dataset and from the compressed dataset transformed by the PCA I have also selected the number of PC I want to keep (they explain almost the 94% of the variance). Now I am struggling with the identification of the original features that are important in the reduced dataset.
How do I find out which feature is important and which is not among the remaining Principal Components after the dimension reduction?
Here is my code:

pandas dataframe columns scaling with sklearn

August 14, 2022 by Magenaut

I have a pandas dataframe with mixed type columns, and I’d like to apply sklearn’s min_max_scaler to some of the columns. Ideally, I’d like to do these transformations in place, but haven’t figured out a way to do that yet. I’ve written the following code that works:

How does the class_weight parameter in scikit-learn work?

August 13, 2022 by Magenaut

I am having a lot of trouble understanding how the class_weight parameter in scikit-learn’s Logistic Regression operates.

Fastest pairwise distance metric in python

August 13, 2022 by Magenaut

I have an 1D array of numbers, and want to calculate all pairwise euclidean distances. I have a method (thanks to SO) of doing this with broadcasting, but it’s inefficient because it calculates each distance twice. And it doesn’t scale well.

scikit-learn cross validation, negative values with mean squared error

August 13, 2022 by Magenaut

When I use the following code with Data matrix X of size (952,144) and output vector y of size (952), mean_squared_error metric returns negative values, which is unexpected. Do you have any idea?

Clustering text documents using scikit-learn kmeans in Python

August 12, 2022 by Magenaut

I need to implement scikit-learn’s kMeans for clustering text documents. The example code works fine as it is but takes some 20newsgroups data as input. I want to use the same code for clustering a list of documents as shown below:

Getting model attributes from pipeline

August 12, 2022 by Magenaut

I typically get PCA loadings like this:

Impute categorical missing values in scikit-learn

August 12, 2022 by Magenaut

I’ve got pandas data with some columns of text type. There are some NaN values along with these text columns. What I’m trying to do is to impute those NaN’s by sklearn.preprocessing.Imputer (replacing NaN by the most frequent value). The problem is in implementation.
Suppose there is a Pandas dataframe df with 30 columns, 10 of which are of categorical nature.
Once I run:

ROC for multiclass classification

August 12, 2022 by Magenaut

I’m doing different text classification experiments. Now I need to calculate the AUC-ROC for each task. For the binary classifications, I already made it work with this code:

Does the SVM in sklearn support incremental (online) learning?

August 12, 2022 by Magenaut

I am currently in the process of designing a recommender system for text articles (a binary case of ‘interesting’ or ‘not interesting’). One of my specifications is that it should continuously update to changing trends.