-
Notifications
You must be signed in to change notification settings - Fork 55
Expand file tree
/
Copy pathVectorizedOperations.html
More file actions
851 lines (790 loc) · 92 KB
/
Copy pathVectorizedOperations.html
File metadata and controls
851 lines (790 loc) · 92 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta content="Topic: Vectorized operations with numpy arrays, Difficulty: Easy, Category: Section" name="description" />
<meta content="vectorize, optimized, calculation, numpy, fast, C routine, MKL, sum, linear algebra, optimized" name="keywords" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>“Vectorized” Operations: Optimized Computations on NumPy Arrays — Python Like You Mean It</title>
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
<link rel="stylesheet" href="../_static/my_theme.css" type="text/css" />
<!--[if lt IE 9]>
<script src="../_static/js/html5shiv.min.js"></script>
<![endif]-->
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script async="async" src="https://www.googletagmanager.com/gtag/js?id=UA-115029372-1"></script>
<script src="../_static/gtag.js"></script>
<script crossorigin="anonymous" integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA=" src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"></script>
<script>window.MathJax = {"tex": {"inlineMath": [["$", "$"], ["\\(", "\\)"]], "processEscapes": true}, "options": {"ignoreHtmlClass": "tex2jax_ignore|mathjax_ignore|document", "processHtmlClass": "tex2jax_process|mathjax_process|math|output_area"}}</script>
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
<script src="../_static/js/theme.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Array Broadcasting" href="Broadcasting.html" />
<link rel="prev" title="Iterating Over Arrays & Array-Traversal Order" href="ArrayTraversal.html" />
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll">
<div class="wy-side-nav-search" >
<a href="../index.html" class="icon icon-home"> Python Like You Mean It
</a>
<div class="version">
1.4
</div>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
<input type="text" name="q" placeholder="Search docs" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
<p class="caption" role="heading"><span class="caption-text">Table of Contents:</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../intro.html">Python Like You Mean It</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_1.html">Module 1: Getting Started with Python</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_2.html">Module 2: The Essentials of Python</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_2_problems.html">Module 2: Problems</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../module_3.html">Module 3: The Essentials of NumPy</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="IntroducingTheNDarray.html">Introducing the ND-array</a></li>
<li class="toctree-l2"><a class="reference internal" href="AccessingDataAlongMultipleDimensions.html">Accessing Data Along Multiple Dimensions in an Array</a></li>
<li class="toctree-l2"><a class="reference internal" href="BasicArrayAttributes.html">Basic Array Attributes</a></li>
<li class="toctree-l2"><a class="reference internal" href="FunctionsForCreatingNumpyArrays.html">Functions for Creating NumPy Arrays</a></li>
<li class="toctree-l2"><a class="reference internal" href="ArrayTraversal.html">Iterating Over Arrays & Array-Traversal Order</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">“Vectorized” Operations: Optimized Computations on NumPy Arrays</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#Basic-Mathematical-Operations-Using-Arrays">Basic Mathematical Operations Using Arrays</a></li>
<li class="toctree-l3"><a class="reference internal" href="#Vectorized-Operations">Vectorized Operations</a></li>
<li class="toctree-l3"><a class="reference internal" href="#NumPy’s-Mathematical-Functions">NumPy’s Mathematical Functions</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#Unary-Functions">Unary Functions</a></li>
<li class="toctree-l4"><a class="reference internal" href="#Binary-Functions">Binary Functions</a></li>
<li class="toctree-l4"><a class="reference internal" href="#Sequential-Functions">Sequential Functions</a></li>
<li class="toctree-l4"><a class="reference internal" href="#Specifying-the-axis-Keyword-Argument-in-Sequential-NumPy-Functions">Specifying the <code class="docutils literal notranslate"><span class="pre">axis</span></code> Keyword Argument in Sequential NumPy Functions</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#Logical-Operations">Logical Operations</a></li>
<li class="toctree-l3"><a class="reference internal" href="#Linear-Algebra">Linear Algebra</a></li>
<li class="toctree-l3"><a class="reference internal" href="#Conclusion">Conclusion</a></li>
<li class="toctree-l3"><a class="reference internal" href="#Links-to-Official-Documentation">Links to Official Documentation</a></li>
<li class="toctree-l3"><a class="reference internal" href="#Reading-Comprehension-Solutions">Reading Comprehension Solutions</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="Broadcasting.html">Array Broadcasting</a></li>
<li class="toctree-l2"><a class="reference internal" href="BasicIndexing.html">Introducing Basic and Advanced Indexing</a></li>
<li class="toctree-l2"><a class="reference internal" href="AdvancedIndexing.html">Advanced Indexing</a></li>
<li class="toctree-l2"><a class="reference internal" href="AutoDiff.html">Automatic Differentiation</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../module_3_problems.html">Module 3: Problems</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_4.html">Module 4: Object Oriented Programming</a></li>
<li class="toctree-l1"><a class="reference internal" href="../module_5.html">Module 5: Odds and Ends</a></li>
<li class="toctree-l1"><a class="reference internal" href="../changes.html">Changelog</a></li>
</ul>
</div>
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="../index.html">Python Like You Mean It</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="navigation" aria-label="Page navigation">
<ul class="wy-breadcrumbs">
<li><a href="../index.html" class="icon icon-home"></a> »</li>
<li><a href="../module_3.html">Module 3: The Essentials of NumPy</a> »</li>
<li>“Vectorized” Operations: Optimized Computations on NumPy Arrays</li>
<li class="wy-breadcrumbs-aside">
<a href="../_sources/Module3_IntroducingNumpy/VectorizedOperations.md.txt" rel="nofollow"> View page source</a>
</li>
</ul>
<hr/>
</div>
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<style>
/* CSS overrides for sphinx_rtd_theme */
/* 24px margin */
.nbinput.nblast.container,
.nboutput.nblast.container {
margin-bottom: 19px; /* padding has already 5px */
}
/* ... except between code cells! */
.nblast.container + .nbinput.container {
margin-top: -19px;
}
.admonition > p:before {
margin-right: 4px; /* make room for the exclamation icon */
}
/* Fix math alignment, see https://github.com/rtfd/sphinx_rtd_theme/pull/686 */
.math {
text-align: unset;
}
</style>
<div class="section" id="“Vectorized”-Operations:-Optimized-Computations-on-NumPy-Arrays">
<h1>“Vectorized” Operations: Optimized Computations on NumPy Arrays<a class="headerlink" href="#“Vectorized”-Operations:-Optimized-Computations-on-NumPy-Arrays" title="Permalink to this headline"></a></h1>
<p>In this section, we will:</p>
<ul class="simple">
<li><p>Define the term vectorization, as it is used in the context of Python/NumPy.</p></li>
<li><p>Prescribe the use of NumPy’s vectorized functions for performing optimized numerical computations on arrays.</p></li>
<li><p>Compare the performance of a simple non-vectorized computation to a vectorized one.</p></li>
<li><p>Describe how unary, binary, and sequential functions are defined on NumPy arrays.</p></li>
<li><p>Provide a brief overview of linear algebra functions and logical operations.</p></li>
</ul>
<div class="section" id="Basic-Mathematical-Operations-Using-Arrays">
<h2>Basic Mathematical Operations Using Arrays<a class="headerlink" href="#Basic-Mathematical-Operations-Using-Arrays" title="Permalink to this headline"></a></h2>
<p>The ND-array can be utilized in mathematical expressions to perform mathematical computations using an array’s entries. In general, NumPy implements mathematical functions such that, when a function acts on an array, the mathematical operation is applied to <em>each</em> entry in the array.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Demonstrating the application of common</span>
<span class="c1"># mathematical operations to a NumPy array</span>
<span class="o">>>></span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mf">3.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">,</span> <span class="mf">5.</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mf">6.</span><span class="p">,</span> <span class="mf">7.</span><span class="p">,</span> <span class="mf">8.</span><span class="p">]])</span>
<span class="c1"># `x ** 2` squares each entry in the array `x`</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">**</span> <span class="mi">2</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">],</span>
<span class="p">[</span> <span class="mf">9.</span><span class="p">,</span> <span class="mf">16.</span><span class="p">,</span> <span class="mf">25.</span><span class="p">],</span>
<span class="p">[</span> <span class="mf">36.</span><span class="p">,</span> <span class="mf">49.</span><span class="p">,</span> <span class="mf">64.</span><span class="p">]])</span>
<span class="c1"># `np.sqrt(x)` takes the square-root</span>
<span class="c1"># of each entry in the array `x`</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">array</span><span class="p">([[</span> <span class="mf">0.</span> <span class="p">,</span> <span class="mf">1.</span> <span class="p">,</span> <span class="mf">1.41421356</span><span class="p">],</span>
<span class="p">[</span> <span class="mf">1.73205081</span><span class="p">,</span> <span class="mf">2.</span> <span class="p">,</span> <span class="mf">2.23606798</span><span class="p">],</span>
<span class="p">[</span> <span class="mf">2.44948974</span><span class="p">,</span> <span class="mf">2.64575131</span><span class="p">,</span> <span class="mf">2.82842712</span><span class="p">]])</span>
<span class="c1"># Slices return arrays, thus you can operate</span>
<span class="c1"># on these too. Add .5 to each entry in row-0</span>
<span class="c1"># of `x`</span>
<span class="o">>>></span> <span class="mf">.5</span> <span class="o">+</span> <span class="n">x</span><span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="p">:]</span>
<span class="n">array</span><span class="p">([</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">1.5</span><span class="p">,</span> <span class="mf">2.5</span><span class="p">])</span>
</pre></div>
</div>
<p>Similarly, mathematical operations performed between two arrays are designed to act on the corresponding pairs of entries between the two arrays:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Demonstrating mathematical operations between two arrays</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mf">3.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">,</span> <span class="mf">5.</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mf">6.</span><span class="p">,</span> <span class="mf">7.</span><span class="p">,</span> <span class="mf">8.</span><span class="p">]])</span>
<span class="o">>>></span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="o">-</span><span class="mf">4.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">3.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">3.</span> <span class="p">],</span>
<span class="o">...</span> <span class="p">[</span><span class="o">-</span><span class="mf">2.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">2.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">1.5</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span><span class="o">-</span><span class="mf">1.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">0.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">0.</span> <span class="p">]])</span>
<span class="c1"># `x + y` will add the corresponding entries of</span>
<span class="c1"># the arrays `x` and `y`</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">+</span> <span class="n">y</span>
<span class="n">array</span><span class="p">([[</span><span class="o">-</span><span class="mf">4.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">2.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">1.</span> <span class="p">],</span>
<span class="p">[</span> <span class="mf">0.5</span><span class="p">,</span> <span class="mf">2.</span> <span class="p">,</span> <span class="mf">3.5</span><span class="p">],</span>
<span class="p">[</span> <span class="mf">5.</span> <span class="p">,</span> <span class="mf">6.5</span><span class="p">,</span> <span class="mf">8.</span> <span class="p">]])</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">*</span> <span class="n">y</span>
<span class="n">array</span><span class="p">([[</span><span class="o">-</span><span class="mf">0.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">3.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">6.</span> <span class="p">],</span>
<span class="p">[</span><span class="o">-</span><span class="mf">7.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">8.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">7.5</span><span class="p">],</span>
<span class="p">[</span><span class="o">-</span><span class="mf">6.</span> <span class="p">,</span> <span class="o">-</span><span class="mf">3.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">0.</span> <span class="p">]])</span>
</pre></div>
</div>
<p>There are also mathematical operations which are designed to operate on sequences of numbers, such as the sum function. NumPy’s sequential functions can act on an array’s entries as if they form a single sequence, or act on subsequences of the array’s entries, according to the array’s axes.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># applying the sequential function, `np.sum`</span>
<span class="c1"># on an array</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mf">3.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">,</span> <span class="mf">5.</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mf">6.</span><span class="p">,</span> <span class="mf">7.</span><span class="p">,</span> <span class="mf">8.</span><span class="p">]])</span>
<span class="c1"># summing over all of the array's entries</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="mf">36.0</span>
<span class="c1"># summing over the rows, within each column</span>
<span class="c1"># of the array</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">array</span><span class="p">([</span> <span class="mf">9.</span><span class="p">,</span> <span class="mf">12.</span><span class="p">,</span> <span class="mf">15.</span><span class="p">])</span>
</pre></div>
</div>
<p>We will use this section to provide a more thorough overview of the various mathematical functions that are provided by NumPy, as well as the behavior of its sequential mathematical operations. However, we must first understand that NumPy performs these “vectorized operations” in a highly-optimized fashion, such that pure Python code can never rival its efficiency. By the end of this section, “vectorized operation” will become a phrase of endearment.</p>
</div>
<div class="section" id="Vectorized-Operations">
<h2>Vectorized Operations<a class="headerlink" href="#Vectorized-Operations" title="Permalink to this headline"></a></h2>
<p>Recall that NumPy’s ND-arrays are <em>homogeneous</em>: an array can only contain data of a single type. For instance, an array can contain 8-bit integers or 32-bit floating point numbers, but not a mix of the two. This is in stark contrast to Python’s lists and tuples, which are entirely unrestricted in the variety of contents they can possess; a given list could simultaneously contain strings, integers, and other objects. This restriction on an array’s contents comes at a great benefit; in “knowing”
that an array’s contents are homogeneous in data type, NumPy is able to delegate the task of performing mathematical operations on the array’s contents to optimized, compiled C code. This is a process that is referred to as <strong>vectorization</strong>. The outcome of this can be a <em>tremendous</em> speedup relative to the analogous computation performed in Python, which must painstakingly check the data type of <em>every</em> one of the items as it iterates over the arrays, since Python typically works with lists
with unrestricted contents.</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Definition</strong>:</p>
<p>In the context of high-level languages like Python, Matlab, and R, the term <strong>vectorization</strong> describes the use of optimized, pre-compiled code written in a low-level language (e.g. C) to perform mathematical operations over a sequence of data. This is done in place of an explicit iteration written in the native language code (e.g. a “for-loop” written in Python).</p>
</div>
<p>Consider, for instance, the task of summing the integers 0-9,999 stored in an array. Calling NumPy’s <code class="docutils literal notranslate"><span class="pre">sum</span></code> function cues optimized C code to iterate over the integers in the array and tally the sum. <code class="docutils literal notranslate"><span class="pre">np.sum</span></code> is therefore a “vectorized” function. Let’s time how long it takes to compute this sum:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="go"># sum an array, using NumPy's vectorized 'sum' function</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">10000</span><span class="p">))</span> <span class="c1"># takes 11 microseconds on my computer</span>
<span class="go">49995000</span>
</pre></div>
</div>
<p>Now let’s compare this to the time required to <em>explicitly</em> loop over the array in Python and tally up the sum. Python is unable to take advantage of the fact that the array’s contents are all of a single data type - it has to check, for every iteration, if it is dealing with an integer, a string, a floating point number, etc, just as it does when iterating over a list. This will slow down the computation massively.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># sum an array by explicitly looping over the array in Python</span>
<span class="c1"># this takes 822 microseconds on my computer</span>
<span class="o">>>></span> <span class="n">total</span> <span class="o">=</span> <span class="mi">0</span>
<span class="o">>>></span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">10000</span><span class="p">):</span>
<span class="o">...</span> <span class="n">total</span> <span class="o">=</span> <span class="n">i</span> <span class="o">+</span> <span class="n">total</span>
<span class="o">>>></span> <span class="n">total</span>
<span class="mi">49995000</span>
</pre></div>
</div>
<p>Timed on my computer, the sum is <strong>over 50 times faster when performed in using NumPy’s vectorized function</strong>! This should make it clear that, whenever computational efficiency is important, one should avoid performing explicit for-loops over long sequences of data in Python, be them lists or NumPy arrays. NumPy provides a whole suite of vectorized functions. In fact, the name of the game when it comes to leveraging NumPy to do computations over arrays of numbers is to exclusively leverage its
vectorized functions. The following computations all invoke vectorized functions:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="go"># multiply 2 with each number in the array</span>
<span class="gp">>>> </span><span class="mi">2</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">])</span>
<span class="go">array([4, 6, 8])</span>
<span class="go"># subtract the corresponding entries of the two arrays</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">10.2</span><span class="p">,</span> <span class="mf">3.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">0.9</span><span class="p">])</span> <span class="o">-</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">8.2</span><span class="p">,</span> <span class="mf">3.5</span><span class="p">,</span> <span class="mf">6.5</span><span class="p">])</span>
<span class="go">array([ 2. , 0. , -7.4])</span>
<span class="go"># Take the "dot product" of the two arrays</span>
<span class="go"># "dot product" means: multiply their corresponding entries and sum the result</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]),</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">]))</span>
<span class="go">6</span>
</pre></div>
</div>
<p>All of the mathematical functions that are introduced in the remainder of this section perform vectorized operations.</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>NumPy provides highly-optimized functions for performing mathematical operations on arrays of numbers. Performing extensive iterations (e.g. via ‘for-loops’) in Python to perform repeated mathematical computations should nearly always be replaced by the use of vectorized functions on arrays. This informs the entire design paradigm of NumPy.</p>
</div>
</div>
<div class="section" id="NumPy’s-Mathematical-Functions">
<h2>NumPy’s Mathematical Functions<a class="headerlink" href="#NumPy’s-Mathematical-Functions" title="Permalink to this headline"></a></h2>
<p>We will now take some time to survey the various types of vectorized mathematical functions that NumPy supplies, and how these mathematical operations, which traditionally are defined on individual numbers, are applied to arrays of numbers. We will look at</p>
<ul class="simple">
<li><p>unary functions: <span class="math notranslate nohighlight">\(f(x)\)</span></p></li>
<li><p>binary functions: <span class="math notranslate nohighlight">\(f(x,y)\)</span></p></li>
<li><p>functions that operate on sequences of numbers: <span class="math notranslate nohighlight">\(f(\{x_i\}_{i=0}^{n-1})\)</span></p></li>
</ul>
<p>These represent a substantial portion of the essential mathematical tools in the NumPy library. An exhaustive list of NumPy’s mathematical functions is available in the <a class="reference external" href="https://numpy.org/doc/stable/reference/ufuncs.html#math-operations">official documentation</a>.</p>
<div class="section" id="Unary-Functions">
<h3>Unary Functions<a class="headerlink" href="#Unary-Functions" title="Permalink to this headline"></a></h3>
<p>A unary function is a mathematical function that only accepts one operand (i.e. argument): <span class="math notranslate nohighlight">\(f(x)\)</span>. NumPy supplies many familiar unary functions:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 64%" />
<col style="width: 36%" />
</colgroup>
<thead>
<tr class="row-odd"><th class="head"><p>Unary Function: <span class="math notranslate nohighlight">\(f(x)\)</span></p></th>
<th class="head"><p>NumPy Function</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(\vert x \vert\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.absolute</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(\sqrt{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.sqrt</span></code></p></td>
</tr>
<tr class="row-even"><td><p><strong>Trigonometric Functions</strong></p></td>
<td></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(\sin{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.sin</span></code></p></td>
</tr>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(\cos{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.cos</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(\tan{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.tan</span></code></p></td>
</tr>
<tr class="row-even"><td><p><strong>Logarithmic Functions</strong></p></td>
<td></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(\ln{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.log</span></code></p></td>
</tr>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(\log_{10}{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.log10</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(\log_{2}{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.log2</span></code></p></td>
</tr>
<tr class="row-even"><td><p><strong>Exponential Functions</strong></p></td>
<td></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(e^{x}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.exp</span></code></p></td>
</tr>
</tbody>
</table>
<p>This is by no means an exhaustive list of the available unary functions, for example the hyperbolic and inverse trigonometric functions are available too. These familiar functions are defined to work on individual numbers (i.e. “scalars”), not sequences of numbers. How, then, does NumPy implement these functions so that they behave in a coherent way when operating on arrays? The answer is that it <strong>maps</strong> the function over the array - applying <span class="math notranslate nohighlight">\(f(x)\)</span> to each element within the array, and
producing a new array as a result (i.e. the input array is not overwritten).</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">])</span>
<span class="c1"># produces array([exp(0.), exp(1.), exp(2.)])</span>
<span class="c1"># x is not overwritten by this; a new array</span>
<span class="c1"># is created</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">array</span><span class="p">([</span> <span class="mf">1.</span> <span class="p">,</span> <span class="mf">2.71828183</span><span class="p">,</span> <span class="mf">7.3890561</span> <span class="p">])</span>
</pre></div>
</div>
<p>This process generalizes to arrays of any dimensionality and shape.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># example of a unary function operating on a 2D array</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span> <span class="p">[</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]])</span>
<span class="o">>>></span> <span class="n">x</span>
<span class="n">array</span><span class="p">([[</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>
<span class="p">[</span><span class="o">-</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]])</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">square</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># equivalent to: `x**2`</span>
<span class="n">array</span><span class="p">([[</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span>
<span class="p">[</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">16</span><span class="p">]])</span>
</pre></div>
</div>
<p>Because slicing returns an array, you can utilize these in mathematical operations as well</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># square column-0 of `x`</span>
<span class="o">>>></span> <span class="n">x</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">]</span> <span class="o">**</span> <span class="mi">2</span>
<span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">9</span><span class="p">])</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>Applying a unary NumPy function, <span class="math notranslate nohighlight">\(f(x)\)</span>, to an N-dimensional array will apply <span class="math notranslate nohighlight">\(f(x)\)</span> elementwise on the array.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Unary Functions</strong></p>
<p>Given the 2D array:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span><span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">14</span><span class="p">,</span> <span class="mi">15</span><span class="p">]])</span>
</pre></div>
</div>
<p>Take the natural-logarithm of the 1st and 3rd element in the 3rd-row of <code class="docutils literal notranslate"><span class="pre">x</span></code>, producing a shape-(2,) result.</p>
</div>
</div>
<div class="section" id="Binary-Functions">
<h3>Binary Functions<a class="headerlink" href="#Binary-Functions" title="Permalink to this headline"></a></h3>
<p>A binary function has the form <span class="math notranslate nohighlight">\(f(x,y)\)</span>. The arithmetic operations are all binary functions:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 50%" />
<col style="width: 25%" />
<col style="width: 25%" />
</colgroup>
<thead>
<tr class="row-odd"><th class="head"><p>Binary Function: <span class="math notranslate nohighlight">\(f(x, y)\)</span></p></th>
<th class="head"><p>NumPy Function</p></th>
<th class="head"><p>Python operator</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(x\cdot y\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.multiply</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">*</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(x\div y\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.divide</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">/</span></code></p></td>
</tr>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(x+y\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.add</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">+</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(x-y\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.subtract</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">-</span></code></p></td>
</tr>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(x^{y}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.power</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">**</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(x \% y\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.mod</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">%</span></code></p></td>
</tr>
</tbody>
</table>
<div class="admonition warning">
<p class="admonition-title fa fa-exclamation-circle"><strong>Recall</strong>:</p>
<p>The “modulo” function (“mod” for short), denoted by <span class="math notranslate nohighlight">\(\%\)</span>, is defined to return the <em>remainder</em> of division: <span class="math notranslate nohighlight">\(5 \% 3 = 2\)</span></p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Using Math Operators on NumPy Arrays</strong>:</p>
<p>It is important to note that NumPy arrays know how to “override” the standard mathematical operators so that they invoke vectorized functions. For example, suppose <code class="docutils literal notranslate"><span class="pre">arr1</span></code> and <code class="docutils literal notranslate"><span class="pre">arr2</span></code> are NumPy arrays; calling <code class="docutils literal notranslate"><span class="pre">arr1</span> <span class="pre">+</span> <span class="pre">arr2</span></code> ends up calling <code class="docutils literal notranslate"><span class="pre">np.add(arr1,</span> <span class="pre">arr2)</span></code> “under the hood”. Thus we can safely use the standard math operators <code class="docutils literal notranslate"><span class="pre">+</span> <span class="pre">-</span> <span class="pre">/</span> <span class="pre">*</span> <span class="pre">**</span></code> between NumPy arrays, and fast vectorized functions will be used for us.</p>
</div>
<p>As indicated in this table, these NumPy functions can be called by invoking the familiar Python math-operators, when used in the context of NumPy arrays.</p>
<p>Here are some other common binary functions:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 68%" />
<col style="width: 32%" />
</colgroup>
<thead>
<tr class="row-odd"><th class="head"><p>Binary Function: <span class="math notranslate nohighlight">\(f(x, y)\)</span></p></th>
<th class="head"><p>NumPy Function</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p><span class="math notranslate nohighlight">\(\max(x, y)\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.maximum</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><span class="math notranslate nohighlight">\(\min(x, y)\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.minimum</span></code></p></td>
</tr>
</tbody>
</table>
<p>There are two cases that we must consider when working with binary functions, in the context of NumPy arrays:</p>
<ol class="arabic simple">
<li><p>When both operands of the function are arrays (of the same shape).</p></li>
<li><p>When one operand of the function is a scalar (i.e. a single number) and the other is an array.</p></li>
</ol>
<p>Similar to the behavior of unary functions applied to an array, a binary function will operate on two same-shape arrays by applying the function to their pairwise elements.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">])</span>
<span class="gp">>>> </span><span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="o">-</span><span class="mf">1.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="o">-</span><span class="mf">2.</span><span class="p">])</span>
<span class="go"># pair-wise addition of elements in `x` and `y`</span>
<span class="gp">>>> </span><span class="n">x</span> <span class="o">+</span> <span class="n">y</span> <span class="c1"># convenient notation for calling `np.add(x, y)`</span>
<span class="go">array([-1., 2., 0.])</span>
</pre></div>
</div>
<p>This process generalizes to arrays of any dimensionality and shape, as long as the two operands have the same shape.</p>
<div class="admonition warning">
<p class="admonition-title fa fa-exclamation-circle"><strong>Important Note</strong>:</p>
<p>You <em>can</em> apply binary NumPy functions to arrays of unlike shapes. For instance, you may want to add a single shape-(2,) array with ten of such arrays, which are stored as a single shape-(10,2) array. This process is known as <strong>broadcasting</strong>, and will be covered in detail in a later section.</p>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># example of a binary function operating on two 2D arrays</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">]])</span>
<span class="o">>>></span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="o">-</span><span class="mi">4</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">]])</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span> <span class="c1"># equivalent to `x + y`</span>
<span class="n">array</span><span class="p">([[</span><span class="mi">11</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>
<span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="mi">4</span><span class="p">]])</span>
<span class="c1"># add column-0 of `x` and row-1 of `y`</span>
<span class="o">>>></span> <span class="n">x</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="n">y</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="p">:]</span>
<span class="n">array</span><span class="p">([</span><span class="mi">6</span><span class="p">,</span> <span class="mi">2</span><span class="p">])</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>Applying a binary NumPy-function, <span class="math notranslate nohighlight">\(f(x,y)\)</span>, to two same-shape arrays will apply <span class="math notranslate nohighlight">\(f(x,y)\)</span> to each of their pairwise elements, producing an array of the same shape as either of the operands.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Binary Functions</strong></p>
<p>Given the 2D array:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span><span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">14</span><span class="p">,</span> <span class="mi">15</span><span class="p">]])</span>
</pre></div>
</div>
<p>Add the four quadrants of <code class="docutils literal notranslate"><span class="pre">x</span></code>, producing a shape-(2, 2) output.</p>
</div>
<p>By now, you may be able to guess NumPy’s behavior when you perform feed a binary function a scalar (i.e. a single number) and an array: the function is applied elementwise on the array, with each application filling one of the function’s arguments, and the single scalar provided everywhere as the other operand. This matches exactly the behavior seen in traditional linear algebra.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="mi">3</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.</span><span class="p">,</span> <span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">])</span> <span class="c1"># convenient notation for calling `np.multiply(3, x)`</span>
<span class="go">array([ 0., 3., 6.])</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">3.</span><span class="p">])</span> <span class="o">**</span> <span class="mi">2</span> <span class="c1"># convenient notation for calling `np.power(x, 2)`</span>
<span class="go">array([ 1., 4., 9.])</span>
</pre></div>
</div>
<p>This process generalizes to an array of any dimensionality and shape.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># examples of a binary function operating on a scalar & an array</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">]])</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">maximum</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
<span class="n">array</span><span class="p">([[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span>
<span class="p">[</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">]])</span>
<span class="c1"># a 3D array of shape-(2, 2, 8)</span>
<span class="o">>>></span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[[</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">,</span> <span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">14</span><span class="p">,</span> <span class="mi">15</span><span class="p">]],</span>
<span class="o">...</span>
<span class="o">...</span> <span class="p">[[</span><span class="mi">16</span><span class="p">,</span> <span class="mi">17</span><span class="p">,</span> <span class="mi">18</span><span class="p">,</span> <span class="mi">19</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">21</span><span class="p">,</span> <span class="mi">22</span><span class="p">,</span> <span class="mi">23</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span><span class="mi">24</span><span class="p">,</span> <span class="mi">25</span><span class="p">,</span> <span class="mi">26</span><span class="p">,</span> <span class="mi">27</span><span class="p">,</span> <span class="mi">28</span><span class="p">,</span> <span class="mi">29</span><span class="p">,</span> <span class="mi">30</span><span class="p">,</span> <span class="mi">31</span><span class="p">]]])</span>
<span class="o">>>></span> <span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">,</span> <span class="p">:,</span> <span class="p">::</span><span class="mi">2</span><span class="p">]</span> <span class="o">*</span> <span class="o">-</span><span class="mi">1</span>
<span class="n">array</span><span class="p">([[</span> <span class="mi">0</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">4</span><span class="p">,</span> <span class="o">-</span><span class="mi">6</span><span class="p">],</span>
<span class="p">[</span> <span class="o">-</span><span class="mi">8</span><span class="p">,</span> <span class="o">-</span><span class="mi">10</span><span class="p">,</span> <span class="o">-</span><span class="mi">12</span><span class="p">,</span> <span class="o">-</span><span class="mi">14</span><span class="p">]])</span>
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>Applying a binary NumPy function, <span class="math notranslate nohighlight">\(f(x,y)\)</span>, to an array and a scalar amounts to “distributing” the function elementwise over the array, everywhere utilizing the scalar as the other operand for the binary function.</p>
</div>
</div>
<div class="section" id="Sequential-Functions">
<h3>Sequential Functions<a class="headerlink" href="#Sequential-Functions" title="Permalink to this headline"></a></h3>
<p>A sequential function expects a variable-length sequence of numbers as an input, and produces a single number as an output: <span class="math notranslate nohighlight">\(f(\{x_i\}_{i=0}^{n-1})\)</span>. The following are some sequential NumPy functions:</p>
<table class="docutils align-default">
<colgroup>
<col style="width: 79%" />
<col style="width: 21%" />
</colgroup>
<thead>
<tr class="row-odd"><th class="head"><p>Sequential Function: <span class="math notranslate nohighlight">\(f(\{x_i\}_{i=0}^{n-1})\)</span></p></th>
<th class="head"><p>NumPy Function</p></th>
</tr>
</thead>
<tbody>
<tr class="row-even"><td><p>Mean of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.mean</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Median of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.median</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Variance of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.var</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Standard Deviation of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.std</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Maximum Value of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.max</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Minimum Value of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.min</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Index of the Maximum Value of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.argmax</span></code></p></td>
</tr>
<tr class="row-odd"><td><p>Index of the Minimum Value of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.argmin</span></code></p></td>
</tr>
<tr class="row-even"><td><p>Sum of <span class="math notranslate nohighlight">\(\{x_i\}_{i=0}^{n-1}\)</span></p></td>
<td><p><code class="docutils literal notranslate"><span class="pre">np.sum</span></code></p></td>
</tr>
</tbody>
</table>
<p>The implementation of sequential NumPy functions is straightforward when working with 1-dimensional arrays:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># demonstrating sequential functions</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">4.</span><span class="p">])</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># can also be invoked as `x.sum()`</span>
<span class="mf">6.</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="c1"># can also be invoked as `x.mean()`</span>
<span class="mf">2.</span>
</pre></div>
</div>
<p>How do these functions behave when they are fed multi-dimensional arrays? By default, NumPy’s sequential functions treat any multidimensional array as if it had been reshaped to a 1-dimensional array. For example:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">]])</span>
<span class="go"># `sum` will treat a multidimensional array</span>
<span class="go"># as if it is a single sequence of numbers, by default</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="go">15</span>
</pre></div>
</div>
<p>This default behavior of sequential NumPy functions can be overwritten by specifying the keyword argument <code class="docutils literal notranslate"><span class="pre">axis</span></code> within the sequential function. This is a very useful and common thing to do. We will carefully study what the axis argument is used for in these and other NumPy functions.</p>
</div>
<div class="section" id="Specifying-the-axis-Keyword-Argument-in-Sequential-NumPy-Functions">
<h3>Specifying the <code class="docutils literal notranslate"><span class="pre">axis</span></code> Keyword Argument in Sequential NumPy Functions<a class="headerlink" href="#Specifying-the-axis-Keyword-Argument-in-Sequential-NumPy-Functions" title="Permalink to this headline"></a></h3>
<p>Let’s delve into the meaning of the <code class="docutils literal notranslate"><span class="pre">axis</span></code> argument by first seeing it in action:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># creating a shape-(3,2) array</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">]])</span>
<span class="c1"># sum over axis-0, within axis-1</span>
<span class="c1"># i.e. sum over the rows, within each column</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span> <span class="c1"># equivalent: x.sum(axis=0)</span>
<span class="n">array</span><span class="p">([</span><span class="mi">6</span><span class="p">,</span> <span class="mi">9</span><span class="p">])</span>
<span class="c1"># sum over axis-1, within axis-0</span>
<span class="c1"># i.e. sum over the columns, within each row</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># equivalent: x.sum(axis=1)</span>
<span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">9</span><span class="p">])</span>
<span class="c1"># negative axis-indices can be used too</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=-</span><span class="mi">1</span><span class="p">)</span> <span class="c1"># equivalent: np.sum(x, axis=1)</span>
<span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">9</span><span class="p">])</span>
<span class="c1"># sum over axis-0 and axis-1</span>
<span class="c1"># i.e. sum the array as if it were a 1D sequence (default behavior)</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span> <span class="c1"># equivalent: x.sum(axis=(0, 1))</span>
<span class="mi">15</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">axis</span></code> argument thus specifies which axis or axes are traversed to produce the input sequences for the sequential function to act on. One sequence is designated for each valid combination of indices of the non-traversed axes. For example, <code class="docutils literal notranslate"><span class="pre">np.sum(x,</span> <span class="pre">axis=0)</span></code> says: “for each of the columns of <code class="docutils literal notranslate"><span class="pre">x</span></code>, sum over its rows”. Thus the following sequences are summed over:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>x[:, 0] -> array([0, 2, 4]) # traverse all rows within column-0
x[:, 1] -> array([1, 3, 5]) # traverse all rows within column-1
</pre></div>
</div>
<p>Thus each column of <code class="docutils literal notranslate"><span class="pre">x</span></code> is summed over, producing a shape-(2,) array containing the result of the two sums. Similarly, <code class="docutils literal notranslate"><span class="pre">np.sum(x,</span> <span class="pre">axis=1)</span></code> produces a shape-(3,) array, which stores the sum along each of the three rows in <code class="docutils literal notranslate"><span class="pre">x</span></code>.</p>
<p>You can also supply <em>multiple</em> axes to the keyword argument by specifying them in a “tuple” of integers (using a list instead of a tuple will <em>not</em> work). <code class="docutils literal notranslate"><span class="pre">np.sum(x,</span> <span class="pre">axis=(0,1))</span></code> cues NumPy to traverse <em>both</em> of <code class="docutils literal notranslate"><span class="pre">x</span></code>’s axes, designating the entirety of <code class="docutils literal notranslate"><span class="pre">x</span></code>’s contents as the sequence, and summing to the single sequence into one number. Recall that this matches the default behavior when no <code class="docutils literal notranslate"><span class="pre">axis</span></code> keyword argument is specified.</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Takeaway</strong>:</p>
<p>All sequential NumPy functions have an <code class="docutils literal notranslate"><span class="pre">axis</span></code> keyword argument that can be specified. <code class="docutils literal notranslate"><span class="pre">axis</span></code> is to be fed a single integer or a tuple of integers, which indicate which array axes are to be traversed to designate the sequences of array data to be operated on. A sequence is generated for each valid combination of indices for the non-traversed axes. By default, <strong>all</strong> of the input-array’s axes are included, thus the entire content of the array is treated as a single sequence.</p>
</div>
<div class="section" id="Understanding-the-axis-argument-with-a-Multi-Dimensional-Array">
<h4>Understanding the <code class="docutils literal notranslate"><span class="pre">axis</span></code> argument with a Multi-Dimensional Array<a class="headerlink" href="#Understanding-the-axis-argument-with-a-Multi-Dimensional-Array" title="Permalink to this headline"></a></h4>
<p>The key to understanding the <code class="docutils literal notranslate"><span class="pre">axis</span></code> keyword argument, when working with multi-dimensional arrays, is to be comfortable with how array traversal works in NumPy. Refer to Section 5 of this module for a refresher on this topic. Consider the following shape-(4,2,3) array:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">24</span><span class="p">)</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">x</span>
<span class="go">array([[[ 0, 1, 2],</span>
<span class="go"> [ 3, 4, 5]],</span>
<span class="go"> [[ 6, 7, 8],</span>
<span class="go"> [ 9, 10, 11]],</span>
<span class="go"> [[12, 13, 14],</span>
<span class="go"> [15, 16, 17]],</span>
<span class="go"> [[18, 19, 20],</span>
<span class="go"> [21, 22, 23]]])</span>
</pre></div>
</div>
<p>We can think of this array as possessing four, 2x3 sheets of numbers. Traversing along axis-0 of <code class="docutils literal notranslate"><span class="pre">x</span></code> amounts to stepping from sheet to sheet, given each valid combination of axis-1 and axis-2 indices. Thus <code class="docutils literal notranslate"><span class="pre">np.mean(x,</span> <span class="pre">axis=0)</span></code> says: “for each combination of row and column, take the mean along the sheets of <code class="docutils literal notranslate"><span class="pre">x</span></code>”. Therefore six distinct sequences within <code class="docutils literal notranslate"><span class="pre">x</span></code> are designated for this sequential function to act on:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>x[:, 0, 0] -> array([ 0, 6, 12, 18]) {mean = 9}
x[:, 0, 1] -> array([ 1, 7, 13, 19]) {mean = 10}
x[:, 0, 2] -> array([ 2, 8, 14, 20]) {mean = 11}
x[:, 1, 0] -> array([ 3, 9, 15, 21]) {mean = 12}
x[:, 1, 1] -> array([ 4, 10, 16, 22]) {mean = 13}
x[:, 1, 2] -> array([ 5, 11, 17, 23]) {mean = 14}
</pre></div>
</div>
<p>Also, notice that the set of valid combinations of axis-1 and axis-2 indices corresponds to the two-by-three grid associated with they layout of a sheet. NumPy will return the six mean values in a shape-(2,3) array, so that the correspondence between each sequence and its mean-value is unambiguous:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="go">array([[ 9., 10., 11.],</span>
<span class="go"> [ 12., 13., 14.]])</span>
</pre></div>
</div>
<div class="admonition warning">
<p class="admonition-title fa fa-exclamation-circle"><strong>Recall</strong>:</p>
<p>Recall that NumPy uses row-major ordering (a.k.a C-ordering) when traversing arrays.</p>
</div>
<p>Suppose we specify two axes, say axis-0 and axis-2; traversing these two axes amounts to stepping along the sheets and columns of <code class="docutils literal notranslate"><span class="pre">x</span></code>, for each axis-1 index. Thus two sequences are produced:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>x[:, 0, :] -> array([ 0, 1, 2, 6, 7, 8, 12, 13, 14, 18, 19, 20]) {mean = 10}
x[:, 1, :] -> array([ 3, 4, 5, 9, 10, 11, 15, 16, 17, 21, 22, 23]) {mean = 13}
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="go">array([ 10., 13.])</span>
</pre></div>
</div>
<p>These observations lead us to the following general result:</p>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Result</strong>:</p>
<p>If <span class="math notranslate nohighlight">\(X\)</span> is an <span class="math notranslate nohighlight">\(N\)</span>-dimensional array, and <span class="math notranslate nohighlight">\(j\)</span> (with <span class="math notranslate nohighlight">\(j \leq N\)</span>) axes are specified in the <code class="docutils literal notranslate"><span class="pre">axis</span></code> keyword argument for a sequential NumPy function, then a <span class="math notranslate nohighlight">\(N-j\)</span>-dimensional array will be produced by this function. The shape of the result will be that of <span class="math notranslate nohighlight">\(X\)</span>, but with the entries associated with those <span class="math notranslate nohighlight">\(j\)</span> axes removed.</p>
</div>
<div class="admonition note">
<p class="admonition-title fa fa-exclamation-circle"><strong>Reading Comprehension: Basic Sequential Functions</strong></p>
<p>A digital image is simply an array of numbers, which instructs a grid of pixels on a monitor to shine light of specific colors, according to the numerical values in that array.</p>
<p>An RGB-image can thus be stored as a 3D NumPy array of shape-<span class="math notranslate nohighlight">\((V, H, 3)\)</span>. <span class="math notranslate nohighlight">\(V\)</span> is the number of pixels along the vertical direction, <span class="math notranslate nohighlight">\(H\)</span> is the number of pixels along the horizontal, and the size-3 dimension stores the red, blue, and green color values for a given pixel. Thus a <span class="math notranslate nohighlight">\((32, 32, 3)\)</span> array would be a 32x32 RBG image.</p>
<p>It is common to work with a collection of images. Suppose we want to store N images in a single array; thus we now consider a 4D shape-<span class="math notranslate nohighlight">\((N, V, H, 3)\)</span> array.</p>
<p>Let’s collect some statistics on a collection of images. For the sake of convenience, let’s simply generate a 4D-array of random numbers as a placeholder for real image data. We will generate 100, 32x32 RGB images:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">images</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
</pre></div>
</div>
<p>Now, compute the following:</p>
<ol class="arabic simple">
<li><p>The average 32x32 RGB image.</p></li>
<li><p>The total sum of all the values in the array.</p></li>
<li><p>The minimum blue value, respective to each image.</p></li>
<li><p>The standard deviation among all the RGB values in all the images, respective to each pixel position (thus you should produce a shape-(32, 32) array of values).</p></li>
<li><p>The maximum red-value in the top-left quadrant, respective to each image.</p></li>
</ol>
</div>
</div>
</div>
</div>
<div class="section" id="Logical-Operations">
<h2>Logical Operations<a class="headerlink" href="#Logical-Operations" title="Permalink to this headline"></a></h2>
<p>NumPy provides <a class="reference external" href="https://numpy.org/doc/stable/reference/routines.logic.html">a suite of logical operations</a> that can operate on arrays. Many of these map logical operations over array entries in the same fashion as NumPy’s mathematical functions. These functions return either a single boolean object, or a boolean-type array.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># check which entries of `x` are less than 6</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span>
<span class="o">...</span> <span class="p">[</span><span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">14</span><span class="p">,</span> <span class="mi">15</span><span class="p">]])</span>
<span class="c1"># returns a boolean-type array</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o"><</span> <span class="mi">6</span> <span class="c1"># equivalent to `np.less(x, 6)`</span>
<span class="n">array</span><span class="p">([[</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">True</span><span class="p">],</span>
<span class="p">[</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">],</span>
<span class="p">[</span><span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">],</span>
<span class="p">[</span><span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">,</span> <span class="kc">False</span><span class="p">]],</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">bool</span><span class="p">)</span>
<span class="c1"># performing a logical comparison between two arrays</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">])</span> <span class="o"><=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">])</span>
<span class="n">array</span><span class="p">([</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">True</span><span class="p">,</span> <span class="kc">False</span><span class="p">],</span> <span class="n">dtype</span><span class="o">=</span><span class="nb">bool</span><span class="p">)</span>
</pre></div>
</div>
<p>Recall from the Essentials of Python module that, due to effect of floating point numbers having limited numerical precision, that you should never rely on two floating point numbers being exactly equal. Rather, you should require that they are sufficiently “close” in value. In this same vein, you ought not check that the entries of two float-type arrays are precisely equal. Towards this end, the function <code class="docutils literal notranslate"><span class="pre">allclose</span></code> can be used to verify that all corresponding pairs of entries between two
arrays are approximately equal in value:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># checking if two arrays match, using `np.allclose`</span>
<span class="o">>>></span> <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">0.1</span><span class="p">,</span> <span class="mf">0.2</span><span class="p">,</span> <span class="mf">0.3</span><span class="p">])</span>
<span class="o">>>></span> <span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">1.</span><span class="p">,</span> <span class="mf">2.</span><span class="p">,</span> <span class="mf">3.</span><span class="p">])</span> <span class="o">/</span> <span class="mi">10</span>
<span class="o">>>></span> <span class="n">np</span><span class="o">.</span><span class="n">allclose</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
<span class="kc">True</span>
</pre></div>
</div>
</div>
<div class="section" id="Linear-Algebra">
<h2>Linear Algebra<a class="headerlink" href="#Linear-Algebra" title="Permalink to this headline"></a></h2>
<p>Lastly, we note that NumPy provides a suite of functions that can perform optimized computations and routines relevant to linear algebra. Included here are functions for performing matrix products and tensor products, solving eigenvalue problems, inverting matrices, and computing vector normalizations. Please refer to the <a class="reference external" href="https://numpy.org/doc/stable/reference/routines.linalg.html">official NumPy documentation</a> for a full listing of these functions.</p>
</div>
<div class="section" id="Conclusion">
<h2>Conclusion<a class="headerlink" href="#Conclusion" title="Permalink to this headline"></a></h2>
<p>NumPy provides users with a wide variety of functions capable of performing operations on arrays of data. Its use of <strong>vectorization</strong> makes these functions incredibly fast, when compared to the analogous computations performed in pure Python. Although the preceding discussion laid out a substantial number of rules for how these functions work, one should not worry about memorizing them. Rather, it is best to apply these functions to arrays of various dimensionality, and build an intuition for
them. You may be pleasantly surprised by how easy it is to get a hang of this material by simply putting it to practice.</p>
</div>
<div class="section" id="Links-to-Official-Documentation">
<h2>Links to Official Documentation<a class="headerlink" href="#Links-to-Official-Documentation" title="Permalink to this headline"></a></h2>
<ul class="simple">
<li><p><a class="reference external" href="https://numpy.org/doc/stable/reference/ufuncs.html#math-operations">Math functions</a></p></li>
<li><p><a class="reference external" href="https://numpy.org/doc/stable/reference/routines.logic.html">Logic functions</a></p></li>
<li><p><a class="reference external" href="https://numpy.org/doc/stable/reference/routines.linalg.html">Linear algebra functions</a></p></li>
</ul>
</div>
<div class="section" id="Reading-Comprehension-Solutions">
<h2>Reading Comprehension Solutions<a class="headerlink" href="#Reading-Comprehension-Solutions" title="Permalink to this headline"></a></h2>
<p><strong>Unary Functions: Solution</strong></p>
<p>Take the natural-logarithm of the 1st and 3rd element in the 3rd-row of <code class="docutils literal notranslate"><span class="pre">x</span></code>, producing a shape-(2,) result.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span><span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">14</span><span class="p">,</span> <span class="mi">15</span><span class="p">]])</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="n">x</span><span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">0</span><span class="p">::</span><span class="mi">2</span><span class="p">])</span>
<span class="go">array([ 2.07944154, 2.30258509])</span>
</pre></div>
</div>
<p><strong>Binary Functions: Solution</strong></p>
<p>Add the four quadrants of <code class="docutils literal notranslate"><span class="pre">x</span></code>, producing a shape-(2, 2) output.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([[</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="mi">7</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">9</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">11</span><span class="p">],</span>
<span class="gp">... </span> <span class="p">[</span><span class="mi">12</span><span class="p">,</span> <span class="mi">13</span><span class="p">,</span> <span class="mi">14</span><span class="p">,</span> <span class="mi">15</span><span class="p">]])</span>
<span class="go"># top-left top-right bottom-left bottom-right</span>
<span class="gp">>>> </span><span class="n">x</span><span class="p">[:</span><span class="mi">2</span><span class="p">,</span> <span class="p">:</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">x</span><span class="p">[:</span><span class="mi">2</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">:]</span> <span class="o">+</span> <span class="n">x</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">:,</span> <span class="p">:</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">x</span><span class="p">[</span><span class="o">-</span><span class="mi">2</span><span class="p">:,</span> <span class="o">-</span><span class="mi">2</span><span class="p">:]</span>
<span class="go">array([[20, 24],</span>
<span class="go"> [36, 40]])</span>
</pre></div>
</div>
<p><strong>Basic Sequential Functions: Solutions</strong></p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">images</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">32</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="go"># 1. The average 32x32 RGB image.</span>
<span class="gp">>>> </span><span class="n">mean_imag</span> <span class="o">=</span> <span class="n">images</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="gp">>>> </span><span class="n">mean_imag</span><span class="o">.</span><span class="n">shape</span>
<span class="go">(32, 32, 3)</span>
<span class="go"># 2. The total sum of all the values in the array.</span>
<span class="gp">>>> </span><span class="n">images</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span>
<span class="go">153422.97903817348</span>
<span class="go"># 3. The minimum blue value, respective to each image.</span>
<span class="go"># the colors are ordered red-blue-green along axis-3</span>
<span class="gp">>>> </span><span class="n">min_blue</span> <span class="o">=</span> <span class="n">images</span><span class="p">[:,</span> <span class="p">:,</span> <span class="p">:,</span> <span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">min</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="gp">>>> </span><span class="n">min_blue</span><span class="o">.</span><span class="n">shape</span>
<span class="go">(100,)</span>
<span class="go"># 4. The standard deviation among all the RGB values in all the images,</span>
<span class="go"># respective to each pixel.</span>
<span class="gp">>>> </span><span class="n">pixel_std_dev</span> <span class="o">=</span> <span class="n">images</span><span class="o">.</span><span class="n">std</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">3</span><span class="p">))</span>
<span class="gp">>>> </span><span class="n">pixel_std_dev</span><span class="o">.</span><span class="n">shape</span>
<span class="go">(32, 32)</span>
<span class="go"># The maximum red-value in the top-left quadrant, respective to each image.</span>
<span class="gp">>>> </span><span class="n">max_red_quad</span> <span class="o">=</span> <span class="n">images</span><span class="p">[:,</span> <span class="p">:</span><span class="mi">16</span><span class="p">,</span> <span class="p">:</span><span class="mi">16</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">axis</span><span class="o">=</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
<span class="gp">>>> </span><span class="n">max_red_quad</span><span class="o">.</span><span class="n">shape</span>
<span class="go">(100,)</span>
</pre></div>
</div>
</div>
</div>
</div>
</div>
<footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
<a href="ArrayTraversal.html" class="btn btn-neutral float-left" title="Iterating Over Arrays & Array-Traversal Order" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
<a href="Broadcasting.html" class="btn btn-neutral float-right" title="Array Broadcasting" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
</div>
<hr/>
<div role="contentinfo">
<p>© Copyright 2021, Ryan Soklaski.</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>