python numpy读取数据_python – numpy数组：快速填充和提取数据

您将数据描述为“坐标列表列表”.从这里我猜你的提取看起来像这样：

for x in points:

for y in x:

for Z in y:

# z is a tuple with GPS coordinates

做这个：

# initially, points is a list of lists of lists

points = itertools.chain.from_iterable(points)

# now points is an iterable producing lists

points = itertools.chain.from_iterable(points)

# now points is an iterable producing coordinates

points = itertools.chain.from_iterable(points)

# now points is an iterable producing individual floating points values

data = numpy.fromiter(points, float)

# data is a numpy array containing all the coordinates

data = data.reshape( data.size/2,2)

# data has now been reshaped to be an nx2 array

itertools和numpy.fromiter都是用c实现的,效率很高.因此,这应该很快进行转换.

问题的第二部分并没有真正说明您想要对数据做什么.索引numpy数组比索引python列表要慢.通过对数据执行大量操作,您可以获得速度.如果不了解您正在使用该数据做什么,很难建议如何解决它.

更新：

我已经使用itertools和numpy完成了所有事情.我对因试图理解此代码而导致的任何脑损伤概不负责.

# firstly, we use imap to call GetMyPoints a bunch of times

objects = itertools.imap(GetMyPoints, xrange(100))

# next, we use itertools.chain to flatten it into all of the polygons

polygons = itertools.chain.from_iterable(objects)

# tee gives us two iterators over the polygons

polygons_a, polygons_b = itertools.tee(polygons)

# the lengths will be the length of each polygon

polygon_lengths = itertools.imap(len, polygons_a)

# for the actual points, we'll flatten the polygons into points

points = itertools.chain.from_iterable(polygons_b)

# then we'll flatten the points into values

values = itertools.chain.from_iterable(points)

# package all of that into a numpy array

all_points = numpy.fromiter(values, float)

# reshape the numpy array so we have two values for each coordinate

all_points = all_points.reshape(all_points.size // 2, 2)

# produce an iterator of lengths, but put a zero in front

polygon_positions = itertools.chain([0], polygon_lengths)

# produce another numpy array from this

# however, we take the cumulative sum

# so that each index will be the starting index of a polygon

polygon_positions = numpy.cumsum( numpy.fromiter(polygon_positions, int) )

# now for the transformation

# multiply the first coordinate of every point by *.5

all_points[:,0] *= .5

# now to get it out

# polygon_positions is all of the starting positions

# polygon_postions[1:] is the same, but shifted on forward,

# thus it gives us the end of each slice

# slice makes these all slice objects

slices = itertools.starmap(slice, itertools.izip(polygon_positions, polygon_positions[1:]))

# polygons produces an iterator which uses the slices to fetch

# each polygon

polygons = itertools.imap(all_points.__getitem__, slices)

# just iterate over the polygon normally

# each one will be a slice of the numpy array

for polygon in polygons:

draw_polygon(polygon)

您可能会发现最好一次处理一个多边形.将每个多边形转换为numpy数组并对其执行向量运算.这样做你可能会获得显着的速度优势.将所有数据放入numpy可能有点困难.

由于你形状奇特的数据,这比大多数numpy东西更难. Numpy几乎假设一个统一形状数据的世界.