1 00:00:00,730 --> 00:00:03,076 Hey, welcome to this new lecture, and I hope you 2 00:00:03,077 --> 00:00:07,748 had an easy time trying to build the geocoding app. 3 00:00:07,749 --> 00:00:10,724 I know that should have been tough, but if you 4 00:00:10,725 --> 00:00:13,733 have gone through a certain point, then that's great too. 5 00:00:14,148 --> 00:00:16,692 In this lecture now, you'll get two things. 6 00:00:16,693 --> 00:00:19,092 The first thing is you'll find the 7 00:00:19,093 --> 00:00:22,106 attachment, in the resources of this lecture. 8 00:00:22,107 --> 00:00:24,340 There you'll find all the files, that 9 00:00:24,341 --> 00:00:26,924 contain the code for this web app. 10 00:00:26,925 --> 00:00:31,874 Specifically, you'll find four directories, so static, templates, uploads 11 00:00:31,875 --> 00:00:35,500 and visual, and also an app.py file. 12 00:00:36,170 --> 00:00:40,510 Now, here I happen to have four app files, 13 00:00:40,511 --> 00:00:43,734 but basically the last one is the final version. 14 00:00:43,735 --> 00:00:46,928 So this is very basic, the ver1 where I 15 00:00:46,929 --> 00:00:50,438 just add a couple of functions in a Python script. 16 00:00:50,439 --> 00:00:52,708 And then I added some more to these functions, and 17 00:00:52,709 --> 00:00:55,700 then some more, and then I added, and then I 18 00:00:55,701 --> 00:00:59,668 eventually added some more code, that completed the application. 19 00:00:59,669 --> 00:01:01,433 So this is what you get. 20 00:01:01,890 --> 00:01:05,001 And you'll find this in a zip file. 21 00:01:05,002 --> 00:01:06,580 So please download that. 22 00:01:07,233 --> 00:01:09,448 Yeah, that's the first thing you'll get in this 23 00:01:09,449 --> 00:01:12,440 lecture, the zip file of the solution code. 24 00:01:12,441 --> 00:01:15,848 And the second thing is, I'll go ahead now and explain how 25 00:01:15,849 --> 00:01:19,308 I approach this problem, and I'll show you the code and what 26 00:01:19,309 --> 00:01:21,960 I built first, and second, and third, and so on. 27 00:01:21,966 --> 00:01:24,650 [No Audio] 28 00:01:24,651 --> 00:01:27,532 So these are the four versions. I have. 29 00:01:27,533 --> 00:01:28,848 ver1, when I added some 30 00:01:28,849 --> 00:01:31,580 code, 2, 3, and 4. 31 00:01:32,270 --> 00:01:35,568 So let's ignore this for a moment, because the first 32 00:01:35,569 --> 00:01:39,510 thing I want to explain is the user interface. 33 00:01:39,511 --> 00:01:42,000 So we'll start from the front end first. 34 00:01:42,610 --> 00:01:43,972 Now, the first thing I did, 35 00:01:43,973 --> 00:01:46,970 though, was creating a directory structure. 36 00:01:46,971 --> 00:01:48,522 So I created an empty static 37 00:01:48,523 --> 00:01:51,914 folder, templates, and uploads, and virtual. 38 00:01:51,915 --> 00:01:53,992 So static and templates are 39 00:01:53,993 --> 00:01:55,700 standard for a Flask application. 40 00:01:56,550 --> 00:01:59,208 And the uploads folder will be just folder where, 41 00:01:59,209 --> 00:02:03,688 I generate these intermediate files, that the application 42 00:02:03,689 --> 00:02:06,434 makes available for download for the users. 43 00:02:06,435 --> 00:02:08,300 You'll see that in just a bit. 44 00:02:08,970 --> 00:02:12,508 I also created this virtual folder there, which 45 00:02:12,509 --> 00:02:16,010 is actually the directory of the virtual environment. 46 00:02:16,011 --> 00:02:19,634 So I made that with Python with the m flag, 47 00:02:19,635 --> 00:02:24,358 virtualenv and virtual for the name of the directory. 48 00:02:24,359 --> 00:02:25,952 So if you execute that, you will get 49 00:02:25,953 --> 00:02:31,210 this virtual directory. With fresh installation of Python, 50 00:02:31,211 --> 00:02:33,950 and pip, and other built in libraries. 51 00:02:34,610 --> 00:02:37,972 And then I went ahead and installed Flask and 52 00:02:37,973 --> 00:02:41,028 I installed Pandas, because you will need Pandas to 53 00:02:41,029 --> 00:02:44,552 read the data, that the user is submitting and 54 00:02:44,553 --> 00:02:47,544 you want to calculate some columns there. 55 00:02:47,545 --> 00:02:51,182 And you also need geopy to geocode 56 00:02:51,183 --> 00:02:54,158 those values, from the Pandas DataFrame. 57 00:02:54,159 --> 00:02:56,332 And, yeah, those are the three third 58 00:02:56,333 --> 00:02:58,760 party libraries, that you need to install. 59 00:02:59,266 --> 00:03:03,564 And once I created these empty directories, then the 60 00:03:03,565 --> 00:03:06,366 second step was to create the user interface. 61 00:03:06,766 --> 00:03:10,112 So what I did was, I created an 62 00:03:10,113 --> 00:03:13,930 index.html template, in the templates folder. 63 00:03:14,670 --> 00:03:18,466 So at first that was quite simple. 64 00:03:18,752 --> 00:03:21,716 So it didn't contain this much code, that 65 00:03:21,717 --> 00:03:25,933 simply contained like a title and some headings there, 66 00:03:27,500 --> 00:03:29,440 and also a basic form. 67 00:03:29,440 --> 00:03:31,433 [No Audio] 68 00:03:31,434 --> 00:03:34,370 And initially, I didn't put anything in the action. 69 00:03:35,030 --> 00:03:38,328 So when the user submits, I had this submit button and 70 00:03:38,329 --> 00:03:42,020 I also had this file type of input in there. 71 00:03:42,950 --> 00:03:47,868 So if you can remember it File, this is a 72 00:03:47,869 --> 00:03:51,468 form, we have this File input and the Submit button. 73 00:03:51,469 --> 00:03:55,378 So these two and I also restricted file submission 74 00:03:55,379 --> 00:03:59,000 to CSV only and yeah, that's the form. 75 00:04:00,110 --> 00:04:03,568 I didn't have this in the beginning, so I won't go 76 00:04:03,569 --> 00:04:06,460 and explain this for now, I'll go through this later. 77 00:04:06,990 --> 00:04:09,078 And yeah, that was the interface. 78 00:04:09,079 --> 00:04:14,270 Later I went and created this main.css file. 79 00:04:14,271 --> 00:04:18,320 So here you find the CSS styling for the web page. 80 00:04:19,010 --> 00:04:21,172 I won't go into this now, 81 00:04:21,173 --> 00:04:23,402 this is quite self explanatory. 82 00:04:23,403 --> 00:04:25,908 So we have done CSS earlier in the 83 00:04:25,909 --> 00:04:28,344 course, and this file is also included in 84 00:04:28,345 --> 00:04:30,462 the resources among the other files. 85 00:04:30,463 --> 00:04:35,134 But I never expected, you have the same styling. 86 00:04:35,135 --> 00:04:36,716 So you may have some different things, 87 00:04:36,717 --> 00:04:39,560 some colors or sizings and so on. 88 00:04:40,250 --> 00:04:43,202 And then you go add and reference 89 00:04:43,203 --> 00:04:46,870 the CSS file to these head tags. 90 00:04:48,170 --> 00:04:51,072 Yeah, basically that's what I did in that second step. 91 00:04:51,073 --> 00:04:54,240 So, first step creating a directory and second step 92 00:04:54,241 --> 00:04:58,294 creating the basic interface and adding the CSS file 93 00:04:58,295 --> 00:05:01,616 into that, if you simply created the HTML code 94 00:05:01,617 --> 00:05:04,592 and then later when you finish our application, you 95 00:05:04,593 --> 00:05:07,860 create a CSS file, that's perfectly okay too. 96 00:05:07,861 --> 00:05:09,284 Or even if you started with a 97 00:05:09,285 --> 00:05:12,100 Python file directly, that's also fine. 98 00:05:12,101 --> 00:05:15,736 Some developers start from the front end and some 99 00:05:15,737 --> 00:05:18,740 start from the back end, that's perfectly okay. 100 00:05:19,350 --> 00:05:21,448 So yeah, once I have the interface, then 101 00:05:21,449 --> 00:05:24,900 I went ahead and created an app file. 102 00:05:25,454 --> 00:05:28,100 So an app.py file. 103 00:05:30,090 --> 00:05:31,692 These four versions are just 104 00:05:31,693 --> 00:05:34,636 for demonstration for this video. 105 00:05:34,637 --> 00:05:37,308 So I made it four different files, to show you the 106 00:05:37,309 --> 00:05:41,056 stages, that I went through to build my final code. 107 00:05:41,057 --> 00:05:43,472 So, ver1, this is what I did, 108 00:05:43,473 --> 00:05:47,638 basically, I imported this libraries. 109 00:05:47,639 --> 00:05:51,550 So from flask, I imported the usual Flask class 110 00:05:51,551 --> 00:05:56,794 and normally render_template method, to return HTML templates. 111 00:05:56,795 --> 00:06:01,140 And I knew that my application involved some 112 00:06:01,141 --> 00:06:05,016 user submission, so some post request, therefore I 113 00:06:05,017 --> 00:06:07,768 included the request method there as well. 114 00:06:07,769 --> 00:06:11,800 And then the send_file method, which is used 115 00:06:11,801 --> 00:06:15,410 to send a file to the browser for downloading. 116 00:06:16,410 --> 00:06:18,018 And yeah, that's about flask. 117 00:06:18,019 --> 00:06:21,004 And then we want to geocode data, right? 118 00:06:21,005 --> 00:06:23,884 So I uploaded from geopy.geocoders, I 119 00:06:23,885 --> 00:06:27,474 imported Nominatim and also pandas. 120 00:06:27,475 --> 00:06:30,352 And once I imported the dependencies, then I went ahead 121 00:06:30,353 --> 00:06:35,968 and created a Flask instance just here, and then three 122 00:06:35,969 --> 00:06:39,440 functions which for now, they don't do anything. 123 00:06:39,441 --> 00:06:41,732 So the first would be the home page as 124 00:06:41,733 --> 00:06:44,292 you see here with this backslash. And I name 125 00:06:44,293 --> 00:06:49,166 that index the function, and I simply return render_template. 126 00:06:50,833 --> 00:06:53,870 And of course here goes index.html. 127 00:06:54,790 --> 00:06:58,450 So this will render the homepage which is index.html. 128 00:06:59,270 --> 00:07:01,368 Then the next thing that you might expect 129 00:07:01,369 --> 00:07:04,152 is, you know the user is, in your web page. 130 00:07:04,153 --> 00:07:07,612 And once the user presses the Choose File button and 131 00:07:07,613 --> 00:07:11,634 they select a file, and then they press Submit. 132 00:07:11,635 --> 00:07:14,556 What you want to do when they press the Submit button? 133 00:07:14,557 --> 00:07:19,308 Is you want now to load that file in Python and 134 00:07:19,309 --> 00:07:21,810 you want to read that as a Pandas DataFrame. 135 00:07:21,811 --> 00:07:25,286 I want to calculate a coordinates column, 136 00:07:25,287 --> 00:07:27,990 where you calculate the latitude and longitude 137 00:07:28,000 --> 00:07:31,033 from the address column using geopy. 138 00:07:31,270 --> 00:07:33,348 And then out of coordinates column, you 139 00:07:33,349 --> 00:07:35,818 want to calculate latitude and longitude. 140 00:07:35,819 --> 00:07:37,748 So we have done this previously in 141 00:07:37,749 --> 00:07:40,794 the course, in the Pandas section. 142 00:07:40,795 --> 00:07:43,188 And then eventually you want to return a 143 00:07:43,189 --> 00:07:45,432 DataFrame, and you want to send that 144 00:07:45,433 --> 00:07:48,590 DataFrame via a render_template method. 145 00:07:48,591 --> 00:07:52,824 You want to send that DataFrame here, down here. 146 00:07:52,825 --> 00:07:55,528 So what we're talking here is, we're talking 147 00:07:55,529 --> 00:07:58,328 about creating a function that does all this. 148 00:07:58,329 --> 00:08:00,504 So it reads the csv file and process 149 00:08:00,505 --> 00:08:03,212 it with Pandas, and then it displays the 150 00:08:03,213 --> 00:08:05,750 table, the DataFrame, as a table. 151 00:08:06,410 --> 00:08:08,300 And I created such a function 152 00:08:08,301 --> 00:08:10,250 and called it success-table. 153 00:08:11,710 --> 00:08:14,976 If you remember, that has to have 154 00:08:14,977 --> 00:08:18,896 this methods parameter, equal to POST because 155 00:08:18,897 --> 00:08:21,230 we are expecting a POST request. 156 00:08:21,890 --> 00:08:23,540 And then we have yet another 157 00:08:23,541 --> 00:08:26,670 function, I call this download-file. 158 00:08:27,570 --> 00:08:31,098 So that's a URL, success-table. 159 00:08:31,099 --> 00:08:35,128 And this is a function name, that we have yet 160 00:08:35,129 --> 00:08:38,909 another URL and another function attached to that URL. 161 00:08:38,910 --> 00:08:40,450 So I call this download. 162 00:08:41,030 --> 00:08:43,758 Again, as I said, every time the user 163 00:08:43,759 --> 00:08:47,566 does something, you want to create a decorator 164 00:08:47,567 --> 00:08:49,452 and a function attached to that. 165 00:08:49,453 --> 00:08:52,268 So first the user will do a submission and you 166 00:08:52,269 --> 00:08:55,986 want to capture that in this function, success-table. 167 00:08:55,987 --> 00:08:59,776 And then, the next thing that the user will do is 168 00:08:59,777 --> 00:09:03,456 it will press the Download button that will be displayed down 169 00:09:03,457 --> 00:09:08,448 here, and that has to trigger another function in Python and 170 00:09:08,449 --> 00:09:12,388 that has to trigger the send_file method in Python, so 171 00:09:12,389 --> 00:09:14,698 that we send the file to the user. 172 00:09:14,699 --> 00:09:16,164 That means we need another 173 00:09:16,165 --> 00:09:18,600 function here called download. 174 00:09:19,090 --> 00:09:21,040 At least that's how I named it. 175 00:09:21,570 --> 00:09:24,088 And yeah, let's go to ver2 now, 176 00:09:24,089 --> 00:09:28,168 so see the difference here, and here, 177 00:09:28,169 --> 00:09:30,376 the index function remains the same. 178 00:09:30,377 --> 00:09:33,752 So that's all we do for index, it's quite simple. 179 00:09:33,753 --> 00:09:37,518 We simply return the initial index.html template. 180 00:09:37,519 --> 00:09:40,360 And then here I've added quite a load of code. 181 00:09:42,010 --> 00:09:44,882 And basically what we have is we check for the request, 182 00:09:44,883 --> 00:09:47,468 If we got a POST request, then I get 183 00:09:47,469 --> 00:09:53,633 the file using request.files and file here is the name, 184 00:09:53,634 --> 00:09:56,666 [No Audio] 185 00:09:56,667 --> 00:09:59,536 of the input in here. 186 00:09:59,537 --> 00:10:01,668 So, yeah, what happens is that when the 187 00:10:01,669 --> 00:10:04,640 user presses the Submit button, this one here, 188 00:10:05,250 --> 00:10:09,380 a URL is triggered, and that URL is 189 00:10:09,381 --> 00:10:13,860 URL for the success_table function. 190 00:10:15,350 --> 00:10:18,792 So this function here, that means this function will be 191 00:10:18,793 --> 00:10:24,392 executed and then we create a DataFrame where, which 192 00:10:24,393 --> 00:10:28,188 will read the file, this file, that we are getting 193 00:10:28,189 --> 00:10:30,812 from the user, from the user form. 194 00:10:30,813 --> 00:10:33,836 And then we do this, Nominatim and 195 00:10:33,837 --> 00:10:36,440 we create a dataframe column there. 196 00:10:37,210 --> 00:10:40,070 You know this from the Pandas section. 197 00:10:40,810 --> 00:10:43,440 We use apply method, and then apply 198 00:10:43,441 --> 00:10:47,318 the geocode, to the Address column. 199 00:10:47,319 --> 00:10:51,604 And then out of the coordinates column, I get 200 00:10:51,605 --> 00:10:55,680 a Latitude column, and I use a lambda function 201 00:10:56,530 --> 00:11:00,522 to extract the latitude from each of the coordinates 202 00:11:00,523 --> 00:11:04,424 rows. And also count for null values there. 203 00:11:04,425 --> 00:11:07,608 And then I don't need this coordinates column, so 204 00:11:07,609 --> 00:11:11,110 I just drop that using the drop method. 205 00:11:11,111 --> 00:11:13,902 And then I convert this to a csv. 206 00:11:13,903 --> 00:11:17,350 So here I'm generating a csv file. 207 00:11:17,351 --> 00:11:20,818 This is the output that we want to give to the user. 208 00:11:20,819 --> 00:11:23,778 And I'm storing this in the uploads folder. 209 00:11:23,779 --> 00:11:24,988 So you just need to pass a 210 00:11:24,989 --> 00:11:29,638 path there, the relative path uploads/geocoded, 211 00:11:29,639 --> 00:11:32,080 that's how I'm naming this. At the moment 212 00:11:32,081 --> 00:11:35,142 later, I'll do something more advanced. 213 00:11:35,143 --> 00:11:39,150 So let's name it geocoded for now, .csv. 214 00:11:39,151 --> 00:11:43,652 And then, what this will do, it will return a 215 00:11:43,653 --> 00:11:47,470 template, so it will return the index.html template. 216 00:11:48,130 --> 00:11:51,588 But then if you look here, let 217 00:11:51,589 --> 00:11:53,780 me open this to another view. 218 00:11:55,670 --> 00:11:58,232 Here we have a division, node, the 219 00:11:58,233 --> 00:12:00,344 Geocoder syntax, that I'm using there. 220 00:12:00,345 --> 00:12:03,736 So I'm including the btn variable and 221 00:12:03,737 --> 00:12:05,800 I'm ignoring when that is missing. 222 00:12:06,330 --> 00:12:08,844 Now this is different from here, up here. 223 00:12:08,845 --> 00:12:13,570 Here I'm returning a HTML as a string, 224 00:12:13,571 --> 00:12:16,402 and here I'm returning an HTML template. 225 00:12:16,403 --> 00:12:18,280 So the syntax is different. 226 00:12:19,210 --> 00:12:22,370 And this is also different from when we use extend. 227 00:12:22,371 --> 00:12:27,078 So when we extending a layout, you extend a layout 228 00:12:27,079 --> 00:12:29,734 when you want that, as soon as the user visits, 229 00:12:29,735 --> 00:12:33,818 let's say the index.html template, the index.html template 230 00:12:33,819 --> 00:12:38,612 will display other HTML templates, so it will extend as 231 00:12:38,613 --> 00:12:41,716 soon as user visits, that index.html page, 232 00:12:41,717 --> 00:12:44,648 so the home page. This is different, because we 233 00:12:44,649 --> 00:12:46,820 don't want this to be displayed right away. 234 00:12:47,510 --> 00:12:49,518 That's why you're saying ignore missing. 235 00:12:49,519 --> 00:12:52,456 So ignore that when it's missing and include it, 236 00:12:52,457 --> 00:12:59,426 only when the user is visiting this success-table 237 00:12:59,427 --> 00:13:02,268 URL. And when the user visits that, 238 00:13:02,269 --> 00:13:04,316 so when the user presses Submit, after 239 00:13:04,317 --> 00:13:07,292 having submitted a file, then you want 240 00:13:07,293 --> 00:13:10,774 to render this download.html template. 241 00:13:10,775 --> 00:13:13,660 So what this contains is a button here. 242 00:13:15,310 --> 00:13:17,870 You want to generate a button there 243 00:13:17,871 --> 00:13:24,110 which should be, you know, we Choose File, Submit. 244 00:13:24,110 --> 00:13:28,290 [No Audio] 245 00:13:28,291 --> 00:13:30,788 So yeah, you get this Download file in 246 00:13:30,789 --> 00:13:33,358 there, and you also get the HTML table, 247 00:13:33,359 --> 00:13:35,890 so the DataFrame as an HTML table. 248 00:13:37,030 --> 00:13:41,560 So the first row, the second row, the first 249 00:13:41,561 --> 00:13:45,932 row, the second row, and yeah, that button now 250 00:13:45,933 --> 00:13:50,268 is attached to a reference, which suggests that when 251 00:13:50,269 --> 00:13:54,802 the user presses download, the url_for the Python 252 00:13:54,803 --> 00:13:57,830 download function will be visited. 253 00:13:58,490 --> 00:13:59,880 So we go here. 254 00:14:00,890 --> 00:14:04,870 This URL of the download function will be visited. 255 00:14:05,930 --> 00:14:08,902 And now what we want to do when the user visits 256 00:14:08,903 --> 00:14:14,350 this URL, is we want to return a send_file method. 257 00:14:14,930 --> 00:14:22,350 Actually this should be uploads/geocoded.csv. 258 00:14:23,250 --> 00:14:25,992 So we want to send to the user the 259 00:14:25,993 --> 00:14:30,152 file that we generated up here earlier in this 260 00:14:30,153 --> 00:14:33,390 function, we want to send that to the user 261 00:14:33,391 --> 00:14:39,080 under the name yourfile.csv. And also as attachments, True. 262 00:14:39,850 --> 00:14:43,346 You know this code from the previous lectures 263 00:14:43,347 --> 00:14:46,898 and yeah, that's about the download.html template. 264 00:14:46,899 --> 00:14:48,780 And yeah, we're basically done. 265 00:14:48,781 --> 00:14:50,496 I mean, you can get away with this 266 00:14:50,497 --> 00:14:52,960 version of the program, and it will be 267 00:14:52,961 --> 00:14:55,580 almost excellent if you went this far. 268 00:14:56,510 --> 00:15:00,304 However, we may have some small problems with this. 269 00:15:00,305 --> 00:15:05,092 The problem would be, you know, if you choose a file, a 270 00:15:05,093 --> 00:15:10,356 csv file that doesn't have an address column inside, so 271 00:15:10,357 --> 00:15:15,358 a column named Address, then what would happen is you'd 272 00:15:15,359 --> 00:15:18,574 be able to load that file as a DataFrame. 273 00:15:18,575 --> 00:15:22,232 And then when you try to create a new 274 00:15:22,233 --> 00:15:25,368 column in the DataFrame called coordinates, and then 275 00:15:25,369 --> 00:15:29,004 you try to access an existing column named Address 276 00:15:29,005 --> 00:15:32,482 in the DataFrame, Python will throw an error 277 00:15:32,483 --> 00:15:34,970 and it will stop the application. 278 00:15:34,971 --> 00:15:38,402 That means when the user visits the success-table 279 00:15:38,403 --> 00:15:42,430 URL, they will be shown an error page. 280 00:15:42,431 --> 00:15:44,630 And that is not very user friendly. 281 00:15:44,631 --> 00:15:47,968 So what you might want to do instead, is you 282 00:15:47,969 --> 00:15:51,800 may add some functionality here that checks, if the 283 00:15:51,801 --> 00:15:55,002 DataFrame, if the csv file entered by the user. 284 00:15:55,003 --> 00:15:57,348 So you want to read the file as a DataFrame, and 285 00:15:57,349 --> 00:16:01,450 then in the DataFrame, check if you have an address column. 286 00:16:01,451 --> 00:16:04,090 If you have one, then you go ahead and do the calculations. 287 00:16:04,091 --> 00:16:05,732 But if you don't, then you 288 00:16:05,733 --> 00:16:08,442 return something to the user. 289 00:16:08,443 --> 00:16:12,884 So instead of returning df.to_html, you return a 290 00:16:12,885 --> 00:16:16,790 message saying that, sorry, you don't have such a column in your csv file. 291 00:16:17,930 --> 00:16:19,372 So, yeah, I'll show you the code 292 00:16:19,373 --> 00:16:21,366 for that in the next lecture, so see you.