Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@jshtaway
jshtaway authored Feb 7, 2019
1 parent effc1c9 commit b5237cf
Showing 1 changed file with 11 additions and 43 deletions.
54 changes: 11 additions & 43 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
# Mission to Mars

![mission_to_mars](Images/mission_to_mars.jpg)

In this assignment, you will build a web application that scrapes various websites for data related to the Mission to Mars and displays the information in a single HTML page. The following outlines what you need to do.
This is a web application that scrapes various websites for data related to the Mission to Mars and displays the information in a single HTML page. The following outlines the details.

## Step 1 - Scraping

Complete your initial scraping using Jupyter Notebook, BeautifulSoup, Pandas, and Requests/Splinter.
Initial scraping used Jupyter Notebook, BeautifulSoup, Pandas, and Requests/Splinter is recommended so you can visualize as you go. It can be moved to Python Flask later.

* Create a Jupyter Notebook file called `mission_to_mars.ipynb` and use this to complete all of your scraping and analysis tasks. The following outlines what you need to scrape.

### NASA Mars News

* Scrape the [NASA Mars News Site](https://mars.nasa.gov/news/) and collect the latest News Title and Paragraph Text. Assign the text to variables that you can reference later.
* Scraped the [NASA Mars News Site](https://mars.nasa.gov/news/) and collected the latest News Title and Paragraph Text.

```python
# Example:
Expand All @@ -23,13 +20,9 @@ news_p = "Preparation of NASA's next spacecraft to Mars, InSight, has ramped up

### JPL Mars Space Images - Featured Image

* Visit the url for JPL's Featured Space Image [here](https://www.jpl.nasa.gov/spaceimages/?search=&category=Mars).

* Use splinter to navigate the site and find the image url for the current Featured Mars Image and assign the url string to a variable called `featured_image_url`.
* Next visited the url for JPL's Featured Space Image [here](https://www.jpl.nasa.gov/spaceimages/?search=&category=Mars).

* Make sure to find the image url to the full size `.jpg` image.

* Make sure to save a complete url string for this image.
* Splinter was used to navigate the site and find the image url for the current Featured Mars Image and assign the url string to the variable `featured_image_url`.

```python
# Example:
Expand All @@ -38,7 +31,7 @@ featured_image_url = 'https://www.jpl.nasa.gov/spaceimages/images/largesize/PIA1

### Mars Weather

* Visit the Mars Weather twitter account [here](https://twitter.com/marswxreport?lang=en) and scrape the latest Mars weather tweet from the page. Save the tweet text for the weather report as a variable called `mars_weather`.
* Next visited the Mars Weather twitter account [here](https://twitter.com/marswxreport?lang=en) was scraped for the latest Mars weather tweet from the page. Saved the weather report in the variable `mars_weather`.

```python
# Example:
Expand All @@ -47,19 +40,12 @@ mars_weather = 'Sol 1801 (Aug 30, 2017), Sunny, high -21C/-5F, low -80C/-112F, p

### Mars Facts

* Visit the Mars Facts webpage [here](http://space-facts.com/mars/) and use Pandas to scrape the table containing facts about the planet including Diameter, Mass, etc.
* Next visited the Mars Facts webpage [here](http://space-facts.com/mars/) Used Pandas to scrape the table containing facts about the planet including Diameter, Mass, etc and convert the data to an html table string.

* Use Pandas to convert the data to a HTML table string.

### Mars Hemispheres

* Visit the USGS Astrogeology site [here](https://astrogeology.usgs.gov/search/results?q=hemisphere+enhanced&k1=target&v1=Mars) to obtain high resolution images for each of Mar's hemispheres.

* You will need to click each of the links to the hemispheres in order to find the image url to the full resolution image.

* Save both the image url string for the full resolution hemisphere image, and the Hemisphere title containing the hemisphere name. Use a Python dictionary to store the data using the keys `img_url` and `title`.

* Append the dictionary with the image url string and the hemisphere title to a list. This list will contain one dictionary for each hemisphere.
* Next visited the USGS Astrogeology site [here](https://astrogeology.usgs.gov/search/results?q=hemisphere+enhanced&k1=target&v1=Mars) to obtain high resolution images for each of Mar's hemispheres.

```python
# Example:
Expand All @@ -75,31 +61,13 @@ hemisphere_image_urls = [

## Step 2 - MongoDB and Flask Application

Use MongoDB with Flask templating to create a new HTML page that displays all of the information that was scraped from the URLs above.

* Start by converting your Jupyter notebook into a Python script called `scrape_mars.py` with a function called `scrape` that will execute all of your scraping code from above and return one Python dictionary containing all of the scraped data.

* Next, create a route called `/scrape` that will import your `scrape_mars.py` script and call your `scrape` function.

* Store the return value in Mongo as a Python dictionary.

* Create a root route `/` that will query your Mongo database and pass the mars data into an HTML template to display the data.

* Create a template HTML file called `index.html` that will take the mars data dictionary and display all of the data in the appropriate HTML elements. Use the following as a guide for what the final product should look like, but feel free to create your own design.
Used MongoDB with Flask templating to create a new HTML page that displays all of the information that was scraped from the URLs above.

![final_app_part1.png](Images/final_app_part1.png)
![final_app_part2.png](Images/final_app_part2.png)

- - -

## Hints

* Use Splinter to navigate the sites when needed and BeautifulSoup to help find and parse out the necessary data.

* Use Pymongo for CRUD applications for your database. For this homework, you can simply overwrite the existing document each time the `/scrape` url is visited and new data is obtained.

* Use Bootstrap to structure your HTML template.

## Copyright
## Requirements

Trilogy Education Services © 2017. All Rights Reserved.
* Spliter, BeautifulSoup, Pymongo, Bootstrap, Flask

0 comments on commit b5237cf

Please sign in to comment.