Ask Ghassem - Recent questions tagged python

Which code has best runtime and why?(the one commented or the other one)

Fri, 02 Sep 2022 14:39:49 +0000

# for key, value in dict.items():
            #     if value >= long:
            #         long = value
            #         long_name = key
            #     if value < short:
            #         short = value
            #         short_name = key
        long = max(dict.values())
        long_name = max(dict, key=dict.get)
        short = min(dict.values())
        short_name = min(dict, key=dict.get)

Kmeans clustering in python - Giving original labels to predicted clusters

Wed, 27 Apr 2022 05:32:54 +0000

I have a dataset with 7 labels in the target variable.

X = data.drop('target', axis=1)
Y = data['target']
Y.unique()

array(['Normal_Weight', 'Overweight_Level_I', 'Overweight_Level_II',
'Obesity_Type_I', 'Insufficient_Weight', 'Obesity_Type_II',
'Obesity_Type_III'], dtype=object)

km = KMeans(n_clusters=7, init="k-means++", random_state=300)
km.fit_predict(X)
np.unique(km.labels_)

array([0, 1, 2, 3, 4, 5, 6])

After performing KMean clustering algorithm with number of clusters as 7, the resulted clusters are labeled as 0,1,2,3,4,5,6. But how to know which real label matches with the predicted label.

In other words, I want to know how to give original label names to new predicted labels, so that they can be compared like how many values are clustered correctly (Accuracy).

ValueError: Length mismatch: Expected axis has 60 elements, new values have 2935849 elements

Fri, 26 Nov 2021 06:09:16 +0000

I'm creating a new data frame with the most used items grouped together. But I got the following error when grouping through ID and items. ValueError: Length mismatch: Expected axis has 60 elements, new values have 2935849 elements.

df = sales_df[sales_df['shop_id'].duplicated(keep=False)]
df['Grouped'] = sales_df.groupby('shop_id')['item_name'].transform(lambda x: ','.join(x))
df2 = df[['shop_id', 'Grouped']].drop_duplicates()

In the aforementioned code, I'm making a data frame with respect to shop id and then grouping through shop items. My objective here is to group items with similar ID.

Can Data Science solve this problem?

Sun, 24 Oct 2021 15:43:11 +0000

So, I live in Brazil, and I have a task for college that I don't know what data science method to use, if at all, to solve it. My idea is the following: We Brazilians have Real (BRL) as currency, and we of course have the dollar quotation value to see "how many Reais a dollar is worth". What I wanted to do was to make a research and see whether the Country News have any influence over this price. So for example, if Bolsonaro, our president, says some dumb stuff, the dollar got up in price, and vice versa. What I wanted to do was collect all dollar values and variance over a set time interval, and try and get webscraping to get the news over some economy sites. Here's my question then: How can I correlate the news with the dollar variance over a set time? Can data science do that? How do I preprocess this, if at all? Do I need to use bag-of-words? At least I heard so... Please help and thank you for reading.

How to remove unwanted Jupyter notebook kernels?

Fri, 30 Oct 2020 17:15:17 +0000

Whener I run Jupyter notebook there are some kernels that do not exist on system and generate errors. How can I remove them?

How to print confusion matrix if I am using stratifiedkfold method?

Thu, 06 Aug 2020 21:41:19 +0000

I am facing the error after importing torch module in python. How can I solve it? The error link is given below

Fri, 31 Jul 2020 14:56:42 +0000

https://ibb.co/MgXvrkL

What is difference between Support vector machine and Support Vector Classification?

Wed, 13 May 2020 20:22:23 +0000

How to share a Jupyter Notebook document on Google Colab?

Thu, 27 Feb 2020 16:17:13 +0000

How can I give access to others to view, comment or edit my Jupyter Notebook on Google Colab?

How to filter a dataframe?

Wed, 25 Dec 2019 05:56:14 +0000

Consider the Pandas DataDrame df below. Filter it appropriately so that it outputs the shown results.

     gh owner language      repo  stars
0  pandas-dev   python    pandas  17800
1   tidyverse        R     dplyr   2800
2   tidyverse        R   ggplot2   3500
3      has2k1   python  plotnine   1450

Expected Output

     gh owner language    repo  stars
0  pandas-dev   python  pandas  17800

How does break, continue and pass work in Python?

Thu, 11 Jul 2019 17:07:58 +0000

What are self and _init_ in Python classes?

Thu, 11 Jul 2019 17:00:39 +0000

What are classes in Python?

Thu, 11 Jul 2019 16:40:21 +0000

What are functions In Python?

Thu, 11 Jul 2019 16:24:29 +0000

What is the difference between Python Arrays and Lists?

Thu, 11 Jul 2019 16:18:22 +0000

Is indentation required in Python? Is Python case-sensitive?

Thu, 11 Jul 2019 16:16:37 +0000

What is type conversion in Python?

Tue, 09 Jul 2019 19:48:13 +0000

What are the local variables and global variables in Python?

Tue, 09 Jul 2019 19:08:20 +0000

What are python modules? Name some commonly used built-in modules in Python?

Tue, 09 Jul 2019 19:02:38 +0000

What is PYTHONPATH in Python?

Tue, 09 Jul 2019 19:01:25 +0000

What is a namespace in Python?

Tue, 09 Jul 2019 18:52:00 +0000

How is memory management in Python?

Tue, 09 Jul 2019 18:41:01 +0000

What is PEP8 in Python?

Tue, 09 Jul 2019 18:39:42 +0000

How is Python an interpreted language?

Tue, 09 Jul 2019 18:37:52 +0000

What type of language is python? Programming or scripting?

Tue, 09 Jul 2019 18:32:30 +0000

What are the key features of Python?

Tue, 09 Jul 2019 18:24:25 +0000

What is the difference between list and tuples in Python?

Tue, 09 Jul 2019 18:20:40 +0000

What are the most important Python libraries for data science?

Mon, 08 Jul 2019 04:42:28 +0000

What are the available libraries for continuous time hidden markov models ?

Fri, 07 Jun 2019 13:27:05 +0000

What are some best features of Python3 in comparison to Python2?

Mon, 20 May 2019 16:11:31 +0000

Python3 and Python2 seems so similar, and the only changes which are obvious are some changes in functions such as print that now needs parentheses. Are there other improvements?

How to open Jupyter notebook files on Windows or Mac without web browser?

Sat, 16 Mar 2019 23:36:01 +0000

Who can write a recursive Python program for Fibonacci sequence 0,1,1,2,3,5,8,...?

Fri, 15 Mar 2019 20:33:17 +0000

Please also check if the fibo(7) = 8

Passing variable length sentences to Tensorflow LSTM

Mon, 11 Feb 2019 05:06:27 +0000

I have a tensorflow LSTM model for predicting the sentiment. I build the model with the maximum sequence length 150. (Maximum number of words) While making predictions, i have written the code as below:

batchSize = 32
maxSeqLength = 150

def getSentenceMatrix(sentence):
    arr = np.zeros([batchSize, maxSeqLength])
    sentenceMatrix = np.zeros([batchSize,maxSeqLength], dtype='int32')
    cleanedSentence = cleanSentences(sentence)
    cleanedSentence = ' '.join(cleanedSentence.split()[:150])
    split = cleanedSentence.split()
    for indexCounter,word in enumerate(split):
        try:
            sentenceMatrix[0,indexCounter] = wordsList.index(word)
        except ValueError:
            sentenceMatrix[0,indexCounter] = 399999 #Vector for unkown words
    return sentenceMatrix

input_text = "example data"
inputMatrix = getSentenceMatrix(input_text)

In the code i'm truncating my input text to 150 words and ignoring remaining data.Due to this my predictions are wrong.

cleanedSentence = ' '.join(cleanedSentence.split()[:150])

I know that if we have lesser length than sequence length we can pad with zero's. What we need to do if we have more length. Can you suggest me the best way to do this. Thanks in advance.

How to find the strength of a P-value against a null hypothesis?

Fri, 07 Dec 2018 22:31:33 +0000

I'm wondering how I can copy files from my computer to the server on Digital Cloud?

Thu, 06 Dec 2018 18:20:56 +0000

I've created a server on Digital Ocean, and employed uWSGI + NGINX to run a Flask application on the Ubuntu server. I need to copy some files from my computer to the server, yet typical methods such as using PuTTY tools (pscp) didn't seem to work properly!

Your advice is highly appreciated.

Merging data using dates

Wed, 05 Dec 2018 12:39:41 +0000

Index

_d.index incident data

DatetimeIndex(['2011-01-01 00:03:43', '2011-01-01 00:03:55',
               '2011-01-01 00:05:03', '2011-01-01 00:04:46',
               '2011-01-01 00:06:07', '2011-01-01 00:06:24',
               '2011-01-01 00:06:48', '2011-01-01 00:08:26',
               '2011-01-01 00:08:56', '2011-01-01 00:09:33',
               ...
               '2016-12-31 23:34:26', '2016-12-31 23:40:22',
               '2016-12-31 23:47:01', '2016-12-31 23:48:07',
               '2016-12-31 23:52:56', '2016-12-31 23:52:49',
               '2016-12-31 23:55:44', '2016-12-31 23:56:01',
               '2016-12-31 23:59:18', '2016-12-31 23:59:52'],
              dtype='datetime64[ns]', name='incident_date_time', length=720370, freq=None)

_dw.index weather data

DatetimeIndex(['2011-01-01 00:00:00', '2011-01-01 01:00:00',
               '2011-01-01 02:00:00', '2011-01-01 02:41:00',
               '2011-01-01 03:00:00', '2011-01-01 04:00:00',
               '2011-01-01 07:00:00', '2011-01-01 08:00:00',
               '2011-01-01 09:00:00', '2011-01-01 10:00:00',
               ...
               '2016-12-31 17:00:00', '2016-12-31 18:00:00',
               '2016-12-31 19:00:00', '2016-12-31 20:00:00',
               '2016-12-31 21:00:00', '2016-12-31 21:19:00',
               '2016-12-31 22:00:00', '2016-12-31 22:10:00',
               '2016-12-31 22:50:00', '2016-12-31 23:00:00'],
              dtype='datetime64[ns]', name='date', length=62021, freq=None)

For the TFS project, I would like to join the weather to the data by date and hour. for example the weather at 2011-01-01 01:00:00 to all the incidents for that hour 2011-01-01 01:00:00 and 2011-01-01 01:10:00 etc. One weather report to many incidents.

What will be probability for the data items within mean +- 1, +-2, +-3 standard deviation?

Wed, 14 Nov 2018 15:10:15 +0000

How much percentage of data items are consumed within mean +-1 , +-2, +-3 standard deviation?

Wed, 14 Nov 2018 15:04:52 +0000

Training neutral net with Tensor Flow

Mon, 12 Nov 2018 20:09:54 +0000

Hello,

I have a jpeg, where each image is around 1080 x 2048 in size and I have around 3000 of those images. I want to train a simple NN on this data. However, I am not sure how to feed the image data in to tensor flow. The examples online have a minst data which is easier since the image size is smaller.

When i tried make an numpy array out of the images I have i get the memory exception.

Any help would be appreciated !

What are Decorators?

Mon, 29 Oct 2018 22:59:50 +0000

What is a Magic class methods ?

Mon, 29 Oct 2018 22:57:15 +0000

If s='Sammy' what is the output of s[2:]?

Mon, 29 Oct 2018 22:54:58 +0000

How do I create comments in my code?

Mon, 29 Oct 2018 22:45:18 +0000

How to discover outliers of a data frame?

Sun, 28 Oct 2018 11:20:51 +0000

How to transform categorical variable into a matrix binary feature?

Sat, 27 Oct 2018 17:23:08 +0000

Which of the following feature transformations would be a good choice to transform a categorical variable into a matrix binary feature?

A. One-hot-encoding

B. Principal Component Analysis (PCA)

C. Box-Cox

D. Informative prior

Hey, anyone knows how to save workspace in jupyter python? I would like to save the variables and functions, etc.

Wed, 24 Oct 2018 13:53:22 +0000

ploting the xticks to be datatime

Sat, 20 Oct 2018 00:47:40 +0000

I have a dataframe like below

                    MT_001	    MT_002	    hour
2012-01-01 00:15:00	3.807107	22.759602	00:15:00
2012-01-01 00:30:00	5.076142	22.759602	00:30:00
2012-01-01 00:45:00	3.807107	22.759602	00:45:00
2012-01-01 01:00:00	3.807107	22.759602	01:00:00
2012-01-01 01:15:00	5.076142	22.048364	01:15:00

I can easily plot it using

data.plot()
plt.show()

However, this doesn't show the xlabel. What if i want to see the graph where the xlabel shows the ticks with 4 hours apart?

What are basic steps for treating missing values?

Fri, 19 Oct 2018 04:08:48 +0000

What are the general steps in data cleaning?

Fri, 19 Oct 2018 03:54:38 +0000

How can you tell a Python library is a good one and useful?

Fri, 19 Oct 2018 03:46:23 +0000

If a big company such as Google behind it, with high probability it is a great library such as TensorFlow. But in general how can I found out?