Problem Set 8 - Documentation & Retrospective

So far in this course we have built things and in the process understood various aspects of how to build web applications.

If we want users to actually use our application, we must provide them with a manual or at the simplest a "how it works" page.

The process of coding is the best way to learn software engineering, and very often we learn just as much by making mistakes as we do by getting things right.

pset: Create a help page for your application. Then take a step back and write about what you learned, what mistakes you made, and a future roadmap for your application.

Learning Objectives:

• Understand how to create a simple and effective help page
• Understand how to learn from your mistakes
• Understand how to communicate your ideas (technical or otherwise) in writing
• Understand how to create a product roadmap

Problem Set 7 - Social

In the previous problem set, we learned how to secure our web application. In this problem set, we will integrate our application to the social web.

pset: take the application you made in problem set 6, and integrate it with one or more social applications, such as Twitter, Facebook, Google+, etc.

Learning Objectives:

• Understand the meaning and importance of the social web
• Understand the meaning of web api's and how to integrate with them
• Understand how we can create value for our users by utilizing the social web

Problem Set 6 - Security

In the previous problem set, we learned about scalability. In this problem set, we are going to learn about security. Since our application will be deployed on the open Internet, it is vulnerable to all sorts of security exploits. In this problem set, we will learn about common security exploits and how we can protect our application against them.

pset: Take the application you created in problem set 5 and add user management and authentication to it. Then try to make commin security exploits on it. Now understand what you can do to protect your application against such exploits.

Learning Objectives:

• Understand user management, authentication, user roles and other related concepts
• Understand the common security exploits possible for web applications
• Understand how to protect your application against them

Problem Set 5 - Scalability

In the previous problem set, we took the application we had developed in problem set 3, and made it more usable. In this problem set we will learn how to create scalable web applications

pset: Take the web application you have with you in problem set 4, and make some requests to it. Measure the time it takes for the application to provide responses. Now seed it with huge data, and again measure the time it took to make responses. Did the response time suffer? Try to make it faster by tweaking your queries. Now try to make it faster by adding a cache. Next simulate thousands of users accessing your application at the same time. Does the response time suffer? Can you make it faster?

Learning Objectives:

• Understand the notion of scalability
• Understand how data and queries affect the response time of an application
• Understand how we can make the application faster by using a cache
• Understand how the number of simultaneous hits can affect the response time of an application
• Understand how we can use clustering to improve the response time
• Find out if using a no-sql database will improve the scalability of your application

Problem Set 4 - Usability

In the previous problem set, we created a simple multi-user web application in which we learned about concurrency and how to avoid problems related to concurrency.

A good web application is one which is not only technically sound but also aesthetically pleasing, and offers a good user experience.

pset: In this problem set, we will take the application made in problem set 3, and understand how to make it more usable

Learning Objectives:

• Understand the notion of usability
• Understand some common usability design patterns
• Learn how to make your web design aesthetically pleasing
• Learn about the importance of clear error messages
• Learn about the importance of internationalization and how to accomplish it in your web framework

Problem Set 3 - Simple multi-user application

In the previous problem set, we created a simple database backed application. We learned, why dynamic applications are better than static ones, and how databases can help us build dynamic web applications.

Now that you have a basic understanding of dynamic web applications, let us start tackling some complex issues. Most web applications are made to be used by multiple users simultaneously. This creates all sorts of concurrency related issues. Think of an airline reservation system. Imagine user A goes to the website to make a reservation and is looking at the available seats. By the time user A decides which seat they want to reserve, user B comes about and reserves that seat. However user A may not yet know of this. It is important that the system does not allow user A to re-reserve that same seat.

pset: In this problem set you can make any kind of a multi-user reservation system. For this exercise, you have to select a web framework on the platform of your choice.

Learning Objectives:

• Understand concurrency related issues in multi-user web applications
• Understand how the database and application server handle concurrency
• Understand how web frameworks help us with the flow of a web application
• Understand how web frameworks help us with templating
• Understand what are database transactions, and why we need them
• Understand how to design a complex data model, which consists of relationships between tables
• Understand different types of SQL joins and the pros and cons of using them

Problem Set 2 - Simple two page database backed application

In the previous problem set (pset 1), we created a simple single page web application. This was a static application, because the data could not be changed.

Real world apps are dynamic where the data changes depending on user activity, or external systems. The question is where will we store this changing data. We could use the file system, but a database is a far better place to store the data.

pset 2: In this problem set, we will create a simple 2 page application.

1. contact book

Learning objectives:

• Understand why we need a dynamic web application
• Understand why a database is the best way to store the data
• Understand the features a database provides (ACID)
• Understand how we connect to the database from our application
• Understand the concepts of UI templates where we will fill in parts with the dynamic data
• Understand page flow
• Understand how to create a simple data model
• Posting data (HTTP GET & POST)
• Deploying a database backed application

Problem Set 1 - One page application

Most of you already know about the Internet and the world wide web (if you are not sure, check out this page, and this page to understand how they differ). The Internet is helping us collaborate in ways which were either impossible or extremely difficult in the past. Even though it may not single handedly eradicate world hunger, poverty, and war, I believe that the Internet can and is playing a role in helping people understand each other, earn money, and live in peace.

In this course you will learn how to make web applications, so you can create a company in your garage and become a millionaire :-)

Now that you understand the difference between the Internet, and the world wide web, it may be a good idea to understand from a user perspective, what makes a good web site. Head over to this page written by Phillip Greenspun to understand more of what makes a great website, and what makes a website a ripe candidate for suck.com. After you are done, check out this page, to understand the financial potential of building Internet applications.

Now that you have finished reading the pages liked to above, you can start preparing for the first activity. In this activity, you have to create a simple one page static website. To do this, you will need a webserver on which you will deploy your website. You can choose any webserver you wish to work with. We have listed a few popular web servers below. Don't worry too much about making the right choice for the web server, since you can always change it for future activities, if you do not like it. Select something and start working.

Web Servers:

• Apache web server
• Lighttpd
• Tomcat (Use this if you intend to use for Java programming the web application)
• Jetty (Use this if you intend to use Java for programming the web application)
• Microsoft IIS (Use this if you intend to use ASP.NET for programming the web application)

Learning Objectives for this section:

1. What is the web
2. What is a web server and a web browser
3. What is the HTTP protocol
4. What is HTML
5. How to install, setup, and start a web server
6. How to point the web server to an application
7. How to connect to the application from a web browser
8. Basic HTML designing and best practices
9. Basic HTTP request / response (also include headers, mime types, error codes)
10. usability 101

Reference:

• Static site development
• Learn HTML in 21 minutes
• HTML Dog

Activity 1: Install the web server of your choice

Create a blog post explaining all that you have done, along with your notes. Submit a link to your blog post in the 'activity submission' form below.

Activity 2: In this activity you have to create a single page website. This page should contain multiple paragraphs, images, and ideally some video embeds (perhaps YOUTube) from the Internet. What follows is a guideline for how you may go about doing this activity.

• Install a web server on your system
• Create a simple web page 'index.html' for your static web page. Use well formed HTML and make it clean and readable
• Start your browser and load the url which will display your page. For most web servers it should be http://localhost:8080 , but it might be different for your server. Check the manual if you cannot view the page
• Create a simple stylesheet style.css and use it for styling your web page. Understand why using a stylesheet is a good idea

Create a blog post explaining all that you have done, along with your notes. Submit a link to your blog post in the 'activity submission' form below.

Activity 3: Install a plugin for your web browser which will let you examine HTTP requests and responses. Look at the request and response when you view your web page. Try and understand what the request means and also understand the request and response headers

Create a blog post explaining all that you have done, along with your notes. Submit a link to your blog post in the 'activity submission' form below.

---

My Notes

Activity1:

Downloaded Apache for Windows (32-bit) from http://httpd.apache.org/download.cgi#apache22

Checked the MD5 sum, and installed it using 'typical' installation on a Win 32 box.

Since this was an msi installer, Apache was installed as a system service. I re-started the service and pointed my browser to localhost port 8080... no luck... tried port 9090... no luck... checked the configuration file and realized that Apache had started on the default port 80. Pointed my browser to the default port and saw the "It Works!" message.

Activity 2:

I found the document root by looking at the configuration file. There is a line which states:

DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs"

So the document root is in the htdocs directory. Opening this directory revealed a file called index.html, which contained the "It Works!" message. This is the file I need to modify for this activity.

I modified index.html, and added some content to it. For the styling part, I had read about less css sometime back, and was keen to try it out. I implemented the style sheet using less css.

---

Activity 3

Since I am using Firefox (ver 6x), I installed the Firebug to examine request and response headers.

I can see the following request headers (below I mention the headers and what they mean):

Host - The domain name of the server (I wonder if it will include the port if server is on a non-standard port)

User-Agent - This is the user agent string. It identifies the browser which made the request.

Accept - Content types that are acceptable

Accept-Language - Acceptable languages

Accept-Encoding - Acceptable encodings

Accept-Charset - Acceptable charsets

Connection - Specifies if the connection will be closed after the response is sent, or whether it will remain persistent

If-Modified-Since - Allows 304 not modified to be returned if the requested content has not been changed.

If-None-Match - Allows 304 not modified to be returned if the requested content has not been changed.

Cache-Control - Can specify to all caching mechanisms from the client to the server, whether this object can be cached or not

I can see the following response headers (below I mention the headers and what they mean):

Date - The date when the response was generated

Server - The server

Last-Modified - When was the last time the response was modified

Etag - Identifier for a specific version of a resource

Accept-Ranges - What partial content range types this server supports

Content-Length - The length of the response body in octets

Content-Type - The mime type of this content

Overview of problem sets

Before designing the problem sets, we first need to outline what concepts the students need to learn in the course.

• Basic Web, HTTP, and HTML concepts (what is the web, what is a web server, what is HTTP, what is HTML & CSS, what do we mean by good HTML design, basics of the request / response cycle)
• Database backed apps 101 (why do we need database backed applications, setting up a database server, designing a small data model, designing a small 2 page web app, page flow, accepting data from the user, displaying data to the user based on the state of the database)
• Concurrency (what are the concurrency issues faced when creating a multi-user application and how to resolve them, database transactions)
• Database backed apps 102 (design a larger web application and understand page flow, usability, designing a more complex data model, and issues of scale, user managemnet, authentication)
• Security (take the app created in the previous problem set and understand the security issues faced by the application, and try to make it more secure)
• Make your own app and learn other aspects like social networking, mashups, etc.