+2 votes
in Programming Languages by (7.6k points)

I have a very large Numpy array of size 750K. In my code, I am trying to check elements in the array using the following logic:

if (vals[0] in mv_list) #mv_list is numpy array

It has been more than 30 minutes and it could not find ~100,000 elements. How can I speed up the search logic?

1 Answer

+1 vote
by (14.8k points)

Checking an element in a list or Numpy array using "in" clause is extremely slow. You should convert Numpy array to Set to speed up the search logic.

Make the following changes in your code and hopefully, it will fix the issue.

mv_list = set(mv_list)

if (vals[0] in mv_list) #mv_list is numpy array

Also, make sure mv_list = set(mv_list) is not inside any loop, otherwise, it will also become slow.

...