Cloud-Based Document Management Framework
Cloud-Based Document Management Framework
Project Report
On
"Cloud Based
Document Management
Framework"
Prepared by
Tirthkumar Shah (17DCE065)
Under the Supervision of
Prof. Khushi Patel
Submitted to
Charotar University of Science & Technology (CHARUSAT)
for the Partial Fulfillment of the Requirements for the
Degree of Bachelor of Technology (B.Tech.)
in Computer Engineering (CE)
for 8th semester B. Tech
Submitted at
CERTIFICATE
This is to certify that the report entitled “Cloud Based Document Management
Framework” is a bonafied work carried out by Tirthkumar Shah (17DCE065) under
the guidance and supervision of Prof. Khushi Patel & Mr Krishnakumar V. G. for the
subject Software Project Major (CE447) of 8th Semester of Bachelor of Technology in
Computer Engineering at Devang Patel Institute of Advance Technology and Research
(DEPSTAR), Faculty of Technology & Engineering (FTE) – CHARUSAT, Gujarat.
To the best of my knowledge and belief, this work embodies the work of candidate
himself, has duly been completed, and fulfills the requirement of the ordinance relating
to the B.Tech. Degree of the University and is up to the standard in respect of content,
presentation and language for being referred by the examiner(s).
Acknowledgement
➢ The Success and final outcome of this project required a lot of guidance and assistance
from many people and I am extremely privileged to have got this all along the completion
of my project. All that I have done is only due to such supervision and assistance and we
would not forget to thank them.
➢ I am extremely thankful to Prof. Khushi Patel and Mr. Krishnakumar V. G. for providing
such a nice support and guidance, although they had busy schedule managing the college
and company work. I owe my deep gratitude to my team members, who took keen interest
on project work and guided me all along, till the completion of my project work by
providing all the necessary information for developing a good project. Also encouraged me
till the completion of my project work.
DEPSTAR(CE) 1
PRJ2020DCE052 ID:17DCE065
Abstract
➢ Today Developers around the world are making efforts to enhance user experience of using
application as well as they are trying to enhance the developer’s workflow of designing
applications to deliver projects and rollout change requests under strict timeline. Stacks can
be used to build web applications in the shortest span of time. The advantage of such
JavaScript stacks helps to build an integrated solution by using only JavaScript. This project
provides the introduction to concept and describes the MEAN stack open source.
➢ The stacks used in web development are basically the response of software engineers to
current demands. They have essentially adopted pre-existing frameworks (including
JavaScript) to make their lives easier. While there are many, MEAN and MERN are just
two of the popular stacks that have evolved out of JavaScript. Both of these stacks are made
up of open-source components and offer an end-to-end framework for building
comprehensive web apps that enable browsers to connect with databases. The common
theme between the two is JavaScript and this is also the key benefit of using either stack.
You can basically avoid any syntax errors or any confusion by just coding in one
programming language, JavaScript. Another advantage of building your next web project
with MEAN or MERN is the fact that you benefit from its enhanced flexibility.
➢ The term MEAN stack refers to a collection of JavaScript based technologies used to
develop web applications. MEAN is an acronym for MongoDB, ExpressJS, Angular and
NodeJS from client to server to database, MEAN is full stack JavaScript.
➢ Overall, using the MEAN stack enables developers to build highly efficient web
applications.
DEPSTAR(CE) 2
PRJ2020DCE052 ID:17DCE065
Table of Contents
Acknowledgement ................................................................................................................................. 1
Abstract.................................................................................................................................................. 2
Table of Figures..................................................................................................................................... 4
Chapter 1 Introduction......................................................................................................................... 5
1.1 Problem Statement...................................................................................................................... 5
1.2 Project Definition ........................................................................................................................ 6
1.3 Objective ...................................................................................................................................... 7
Chapter 2 Project Planning.................................................................................................................. 8
2.1 Gantt Chart ................................................................................................................................. 9
Chapter 3 Technologies and Tools Used: .......................................................................................... 10
3.1 Technologies .............................................................................................................................. 10
3.2 Tools ........................................................................................................................................... 12
Chapter 4 Requirements .................................................................................................................... 13
4.1 Functional Requirements ......................................................................................................... 13
4.2 Non-Functional Requirements ................................................................................................. 16
4.3 Hardware Requirements .......................................................................................................... 17
Chapter 5 Implementation ................................................................................................................. 18
5.1 Frontend..................................................................................................................................... 18
5.2 Backend...................................................................................................................................... 26
5.2.1 Key Topics .............................................................................................................................. 27
5.3 Code Management- Azure DevOps ......................................................................................... 33
5.3.1 Setup of Azure DevOps ......................................................................................................... 34
5.4 Docker Containerization .......................................................................................................... 36
Chapter 6 UML Diagrams ................................................................................................................. 39
6.1 System Design ............................................................................................................................ 39
6.2 Use Case Diagram ..................................................................................................................... 40
6.3 Class Diagram ........................................................................................................................... 41
6.4 Activity Diagram ....................................................................................................................... 42
Chapter 7 Conclusion ......................................................................................................................... 43
Chapter 8 References .......................................................................................................................... 44
DEPSTAR(CE) 3
PRJ2020DCE052 ID:17DCE065
Table of Figures
Figure 1 Gantt Chart................................................................................................................................ 9
Figure 2 Frontend Technologies ........................................................................................................... 10
Figure 3 Node,Mongo,Express .............................................................................................................. 11
Figure 4 Tools ........................................................................................................................................ 12
Figure 5 Upload Functionality ............................................................................................................... 13
Figure 6 Delete Functionality ................................................................................................................ 14
Figure 7 Preview File ............................................................................................................................. 15
Figure 8 Folder Creation ....................................................................................................................... 15
Figure 9 Registration ............................................................................................................................. 18
Figure 10 Main Window-Read Access................................................................................................... 18
Figure 11 Search Filter .......................................................................................................................... 19
Figure 12 Refresh Feature ..................................................................................................................... 19
Figure 13 Main Window-Write Access.................................................................................................. 20
Figure 14 Creation of Folder ................................................................................................................. 20
Figure 15 Upload Files Dialog ................................................................................................................ 21
Figure 16 Drag and Drop Feature.......................................................................................................... 21
Figure 17 Window of Uploading Files ................................................................................................... 22
Figure 18 Navigate through Folders...................................................................................................... 22
Figure 19 Administrator Interface ........................................................................................................ 23
Figure 20 Delete Feature ...................................................................................................................... 23
Figure 21 Download Files ...................................................................................................................... 24
Figure 22 Logout Window ..................................................................................................................... 25
Figure 23 Implementation Diagram ...................................................................................................... 26
Figure 24 Node Module-Multer S3 ....................................................................................................... 27
Figure 25 Backend-Project Structure .................................................................................................... 27
Figure 26 Application Programming Layer............................................................................................ 28
Figure 27 Robo3T Database Snapshot .................................................................................................. 29
Figure 28 Metadata of Amazon S3........................................................................................................ 30
Figure 29 Postman-API Testing ............................................................................................................. 31
Figure 30 Deleted File Action ................................................................................................................ 32
Figure 31 Azure DevOps ........................................................................................................................ 33
Figure 32 Azure Setup ........................................................................................................................... 34
Figure 33 Project Home Window .......................................................................................................... 34
Figure 34 Azure Repository ................................................................................................................... 35
Figure 35 Commit History ..................................................................................................................... 35
Figure 36 Docker Introduction .............................................................................................................. 36
Figure 37 Docker Toolbox ..................................................................................................................... 37
Figure 38 Containers ............................................................................................................................. 37
Figure 39 Docker Compose Up ............................................................................................................. 38
Figure 40 Multi-Containers ................................................................................................................... 38
Figure 41 System Design ....................................................................................................................... 39
Figure 42 Use Case ................................................................................................................................ 40
Figure 43 Class Diagram ........................................................................................................................ 41
Figure 44 Activity Diagram .................................................................................................................... 42
DEPSTAR(CE) 4
PRJ2020DCE052 ID:17DCE065
Chapter 1 Introduction
DEPSTAR(CE) 5
PRJ2020DCE052 ID:17DCE065
➢ The system should be able to upload files to a specific S3 bucket and ask for user input
for certain metadata fields such as “Document Type” and “Remarks”.
➢ Users should also be able to view, download and delete files stored in the bucket.
➢ Optional features include a directory view to create one or more layers of folders in
order to store files in an organized manner. One should also be able to navigate through
this directory with ease.
DEPSTAR(CE) 6
PRJ2020DCE052 ID:17DCE065
1.3 Objective
DEPSTAR(CE) 7
PRJ2020DCE052 ID:17DCE065
➢ Sprints end with a demo meeting where the functionality is shown to the product
owner, followed by a retrospective meeting where the team discusses what went well
and what needs improvement in their process.
DEPSTAR(CE) 8
PRJ2020DCE052 ID:17DCE065
DevOps Pipeline
Deployment
DEPSTAR(CE) 9
PRJ2020DCE052 ID:17DCE065
3.1 Technologies
➢ Frontend
• Angular 8:
Angular is a platform and framework for building single-page client
applications using HTML and TypeScript. Angular is written in TypeScript. It
implements core and optional functionality as a set of TypeScript libraries that
you import into your apps.
• JavaScript (ES6)
JavaScript, often abbreviated as JS, is a programming language that conforms
to the ECMAScript specification. JavaScript is high-level, often just-in-time
compiled, and multi-paradigm. It has curly-bracket syntax, dynamic typing,
prototype-based object-orientation, and first-class functions.
• H TML5, CSS3
HTML (the Hypertext Markup Language) and CSS (Cascading Style Sheets) are two
of the core technologies for building Web pages. HTML provides the structure of the
page, CSS the (visual and aural) layout, for a variety of devices.
DEPSTAR(CE) 10
PRJ2020DCE052 ID:17DCE065
➢ Backend
• NodeJs
Node.js is an open-source, cross-platform, back-end JavaScript runtime
environment that runs on the V8 engine and executes JavaScript code outside
a web browser.
• Mongo DB
MongoDB is a source-available cross-platform document-oriented database
program. Classified as a NoSQL database program, MongoDB uses JSON-like
documents with optional schemas.
• Azure DevOps
Azure DevOps Server is a Microsoft product that provides version control,
reporting, requirements management, project management, automated builds,
testing and release management capabilities. It covers the entire application
lifecycle, and enables DevOps capabilities.
Figure 3 Node,Mongo,Express
DEPSTAR(CE) 11
PRJ2020DCE052 ID:17DCE065
3.2 Tools
➢ Code Editor
• Visual Studio code
Visual Studio Code is a freeware source-code editor made by Microsoft for
Windows, Linux and macOS. Features include support for debugging, syntax
highlighting, intelligent code completion, snippets, code refactoring, and
embedded Git.
➢ Design
• Adobe XD
Adobe XD is a vector-based user experience design tool for web apps and
mobile apps, developed and published by Adobe Inc. It is available for macOS
and Windows.
Figure 4 Tools
DEPSTAR(CE) 12
PRJ2020DCE052 ID:17DCE065
Chapter 4 Requirements
1. Upload File
➢ The main functionality of system is to upload the files in the Blob storage. Here, as a
blob storage we have used Amazon S3 (Simple storage Service).
➢ If the user has write access or user is administrator, then and only then user can upload
files.
DEPSTAR(CE) 13
PRJ2020DCE052 ID:17DCE065
2. Delete File
➢ This functionality focuses on deletion of file from the storage.
➢ For delete action, user have to access the ownership of file, otherwise it cannot delete
the file from the system except it has administrator power.
➢ Here, instead of deleting the file permanently, file will be moved to another blob
storage and from there it will be delete after 7 days.
DEPSTAR(CE) 14
PRJ2020DCE052 ID:17DCE065
3. Download/View File
➢ The system has designed in such a way that every user has read access, so user will be
able to download and view the files.
4. Create Folder
➢ With the help of this feature, user will be able to create folder in the system. Also, we
have programmed nested folder. So, user can make unlimited folder inside the folder.
➢ Here, User can create folder only if it has write access of the system.
5. Delete Folder
➢ In this functionality, only folder owner can delete the folder himself/herself.
➢ Another user will not be able to delete the folder, unless it has administrator access.
6. User Registration/Login
➢ Here, users can register themselves and got access to the system.
DEPSTAR(CE) 15
PRJ2020DCE052 ID:17DCE065
1. Highly Scalable
➢ In today’s world system should be run all the time, consider that point in mind, we
created the cloud architecture in such a manner that I will be fault tolerance.
➢ Even if the system crash at some point, it will be automatically online within few
minutes.
➢ Here, we implemented Highly Scalable cloud architecture.
2. Highly Available
➢ First of all, highly available means system’s available time.
➢ We can calculate the Availability of system from below formula:
Availability= (Available Time) / (Total Time (24 hours)) X 100
➢ Our goal is to maintain system’s availability as high as possible.
DEPSTAR(CE) 16
PRJ2020DCE052 ID:17DCE065
➢ Here, mentioned below specifications are needed for smooth execution of the project.
➢ Windows 10
➢ Intel i7-7500u
➢ 8 GB RAM
➢ 256 GB SSD
DEPSTAR(CE) 17
PRJ2020DCE052 ID:17DCE065
Chapter 5 Implementation
5.1 Frontend
1. Register/Login Page:
Figure 9 Registration
DEPSTAR(CE) 18
PRJ2020DCE052 ID:17DCE065
3. Search Function
4. Refresh Button
DEPSTAR(CE) 19
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 20
PRJ2020DCE052 ID:17DCE065
7. Upload File/Files
DEPSTAR(CE) 21
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 22
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 23
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 24
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 25
PRJ2020DCE052 ID:17DCE065
5.2 Backend
Implementation
➢ In this system, used Mongo DB Database for User storage and access permission, and
also for authentication purpose.
➢ The main aim of the system is to upload files, which are stored on Blob storage.
➢ Here, project uses Amazon S3 (Simple Storage Service) for storing and retrieving all files
and folders.
➢ In this project, created middle layer of APIs (Application Programming Interfaces) to
connect blob storage and client.
➢ Also, created Upload functionality of files and folders which will make changes from
User Interface to Amazon S3, with the help of Amazon Web Services SDK (Software
Development Kit) and Nodejs.
➢ For Code management, used GitHub repository functionalities.
➢ Also, implemented CI/CD (Continuous Integration and Continuous Development)
pipeline to enhance the work flow, using Azure DevOps.
➢ Here, API is the key implementation of backend, which I will be explaining later in the
report.
DEPSTAR(CE) 26
PRJ2020DCE052 ID:17DCE065
1. Multer S3:
2. Main Structure:
DEPSTAR(CE) 27
PRJ2020DCE052 ID:17DCE065
➢ System’s main heart is the APIs, that upload, create, read and deletes the data and
files.
➢ Created and Implemented API layer between Client and Amazon S3 for data sending
and receiving.
➢ Also, these APIs can be used by company in another project also.
DEPSTAR(CE) 28
PRJ2020DCE052 ID:17DCE065
4. Database
➢ As mentioned above, this project uses MongoDB for the database part.
➢ System has only one database, which is of user model as of now, as the files and files
related metadata are stored in Amazon S3.
➢ User model consists of these many fields, which is mentioned in the below image.
DEPSTAR(CE) 29
PRJ2020DCE052 ID:17DCE065
5. Amazon S3 Metadata:
➢ The system uploads the 7 types of metadata to S3, which is visible in the below
image.
➢ While user is uploading the file, we fetch the user’s name and the user id from the
database, also fetch the remarks and description data from the user interface, together
all these things will be stored in S3 object
➢ Also, there is one unique identifier field is there, named docid, which is auto
generated using uuid, Nodejs module.
DEPSTAR(CE) 30
PRJ2020DCE052 ID:17DCE065
6. Postman:
DEPSTAR(CE) 31
PRJ2020DCE052 ID:17DCE065
7. Delete Functionality:
➢ System whenever calls the delete API, it will move the particular file to another S3
bucket.
➢ From that S3 bucket, it will be automatically deleting after 7 days, using
programmatic script.
➢ Here is the snapshot of that script:
DEPSTAR(CE) 32
PRJ2020DCE052 ID:17DCE065
3. Pipeline
Azure Pipelines is the service to automate builds and deployments with tasks and
reproducible steps for consistency of CI/CD pipeline stages.
4. Test Plans
Azure Test Plans is the service to create and execute test plans, improving the quality
of the code and validating results throughout the pipeline, with end-to-end traceability.
5. Artifacts
Azure Artifacts is the service for package repository.
DEPSTAR(CE) 33
PRJ2020DCE052 ID:17DCE065
1. Project Window:
➢ This is the main landing page of the Azure DevOps.
➢ From here, we can make the projects for the various teams.
➢ Our project tab is in the below image, named DocR.
DEPSTAR(CE) 34
PRJ2020DCE052 ID:17DCE065
3. Repos Tab:
➢ This window consists of all the source code of the project.
➢ Here, it displays all the commits, pushes and pull requests.
➢ In our project, we migrate our GitHub repo to Azure DevOps repos.
4. Commits Window:
➢ It shows the status of the commit from respective branches of the repos.
➢ Also, we can search the commit id from the search bar to see the detailed changes in
case of error handling.
DEPSTAR(CE) 35
PRJ2020DCE052 ID:17DCE065
➢ Docker is an open-source DevOps tool designed to help developers and operations guys
to streamline application development and deployment.
➢ By dockerizing an application means deploying and running an application using
containers.
➢ Containers allow a developer to package up an application with all of the parts it needs,
such as libraries and other dependencies, and deploy it as one package.
➢ By doing so application can be deployed on any target environment/machine regardless of
any customized settings that machine might have that could differ from the machine used
for writing and testing the code.
DEPSTAR(CE) 36
PRJ2020DCE052 ID:17DCE065
1. Docker Toolbox
➢ Docker Toolbox is an installer for quick setup and launch of a Docker environment on
Mac and Windows systems.
➢ From here, we can interact with the docker client.
➢ Also, we can execute docker related command here.
2. Docker Containers
➢ For this project, I made 3 containers, which are Frontend, Backend and Database.
➢ First of all, we have to create DockerFile for each, in that we fetch the image of
particular framework from the docker hub.
➢ After that, declare the commands that are necessary to run the system.
➢ Using docker ps command, we can see the running containers of the project.
Figure 38 Containers
DEPSTAR(CE) 37
PRJ2020DCE052 ID:17DCE065
3. Docker Compose
➢ Compose is a tool for defining and running multi-container Docker applications. With
Compose, you use a YAML file to configure your application’s services. Then, with a
single command, you create and start all the services from your configuration.
➢ Here, in this project, we combined two Docker files which are frontend and backend
into one docker compose file.
➢ From docker compose file, one can run the multiple containers with a single command.
Figure 40 Multi-Containers
DEPSTAR(CE) 38
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 39
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 40
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 41
PRJ2020DCE052 ID:17DCE065
DEPSTAR(CE) 42
PRJ2020DCE052 ID:17DCE065
Chapter 7 Conclusion
➢ This project will simplify the work of the employees of the respective business unit.
➢ It gave us exposure to Full Stack Development and also give us understanding that how to
work in a team for the project, which will be extremely useful in industry.
➢ A robust file repository system that supports simultaneous uploads of multiple files,
structuring of files into infinite layers of folders/subfolders, and access control using a
user login system that also manages privileges and permitted actions of users.
➢ A Docker image of the application that allows it to be run on any operating system in the
way it was intended.
DEPSTAR(CE) 43
PRJ2020DCE052 ID:17DCE065
Chapter 8 References
➢ https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html
➢ https://angular.io/guide/architecture
➢ https://expressjs.com/en/5x/api.html
➢ https://docs.mongodb.com/manual/crud/
➢ https://www.tcs.com/
➢ https://docs.microsoft.com/en-us/azure/devops/?view=azure-devops
DEPSTAR(CE) 44