Benevity PDF Generator
project / March 16th, 2024
Background
This is a historical giving PDF generator application for Benevity that generates of thousands of PDF reports from CSVs, images, and text inputs. Users can upload CSV files containing donor information, manually adjust the column widths of the table in a live generated PDF preview, send an email containing a link to the ZIP file containing all PDFs generated for each user in the CSV data, and load and reuse previously used inputs.
This application is the industry-sponsored team design Capstone project for the University of Calgary Software Engineering Masters program, built with teammates Sam Rainbow and Samuel Sofela over a span of three months from May 2023 to August 2023.
Purpose
For the industry-sponsored team design Capstone project for the University of Calgary Software Engineering Masters program, our team chose the Historical Giving & Volunteering PDF Generator for Benevity.
Benevity is a Calgary-based company that provides platforms for clients to manage and track charitable giving and volunteering of their employees. New clients of Benevity are often transitioning from another giving or volunteering vendor, and require the historical activity of their employee donors or volunteers to be made available.
Previously, an existing internal application was used by Benevity to generate PDF documents that contain the historical activity of an employee donor or volunteer.
The motivation and objective behind this project is to create a new full-stack PDF generator application that can replace the existing application used by Benevity and will address numerous bugs, issues, and missing features that include:
The user is required to adjust column widths in the PDF for each CSV field without a preview or visual aid for guidance. The user has to adopt a trial-and-error approach of generating the PDFs and opening one to see the adjustments they have to make. This can be time-consuming and tedious.
A blurry banner image of the Benevity logo being displayed as the header of each generated PDF file.
The user is required to upload an HTML template when they could instead submit a disclaimer text for the PDF footer, since that is the only part of the template that is different for each generation.
The user is required to manually sort the Employee IDs in the CSV file before uploading the CSV.
Like in the previous application it is replacing, the user will be required to enter input details for the PDF(s) and document mapping file that is generated and eventually compressed in a ZIP file that is emailed to the user for download. The document mapping file is a CSV that shows which PDFs are associated with each employee ID. Once the user has submitted a valid report title, file name prefix, company logo (watermark image), banner image, HTML template for the PDF(s), and a CSV containing the activity of an employee donor or volunteer, the user will be taken to the column width interface.
The column width interface is where the user adjusts the widths of the columns that are extracted from the activity CSV and are displayed in the PDF. Once the user has submitted the column widths, the ZIP file is created and emailed to the specified destination email address.
Concepts and Tools Used
Vue
JavaScript
Java Spring Boot
HTML
CSS
MySQL
AWS
Docker
Git
Process
High-Level Conceptual Design
Following our initial meeting with Benevity and gathering the requirements that the new application should contain, roles were defined for each team member. My primary responsibilities were to develop the front-end application using the Vue framework in JavaScript, which I had never used before. In addition to learning about our respective tools and technologies during the first month of the project, we also started to devise a high-level conceptual design of our solution where we finalized the technology stack we would use.
The user will input details and upload a CSV containing donations or volunteer amounts into Benevity’s web page. The web page will be written in Vue and interact with the back-end application written in Java and Spring Boot. The back-end will interface with a MySQL database in order to store the inputs from the user used to produce the report so that they can be regenerated. The back-end will also interface with Amazon Web Services (AWS) in order to store generated PDFs that can be sent to an email address or re-downloaded at a later time.
Additionally, we defined the main use-cases the application will have:
Next, we illustrated the the system activity workflow of our application:
Front-end Methodology
Once the high-level design is completed, each team member switches their focus to their respective tasks. The process for devising a front-end solution begins by utilizing the project requirements to brainstorm and create wireframes using the Miro visual collaboration platform, in order to visualize the look and flow of the application.
This is a rough wireframe for the PDF and document mapping inputs form that is displayed on the inputs route when the user opens the application. As per the requirements, an input has been added to specify disclaimer text for the footer of each PDF.
Once the next button at the bottom is clicked, the logic for the component will validate the form inputs. If all inputs are valid, this form will be hidden and the component below will be displayed.
This a wireframe for the column widths form that is displayed on the inputs page after the PDF and document mapping inputs form is hidden. Here, the column names shown will be extracted from the CSV and will be dynamically rendered in the form. A change from the previous application includes the addition of checkboxes that will give the user an option to include or exclude the column from the PDF - if a column is excluded, the column width input for that column name will be disabled and be excluded from total width calculation.
Another addition in this wireframe is the PDF preview panel below the column width inputs, where if a user clicks a button, a random PDF is chosen from the list of generated PDFs and embedded in the page for the user to view. At the bottom of the wireframe, the user can click a checkbox to indicate that they wish to save the configuration inputs they entered in this form and the previous form.
This shows a wireframe for the configurations list page. Here, a list of all saved configurations are displayed to the user. Each configuration item will display the name, creator, and description and contain buttons to load the configuration in the input forms or delete the configuration from the list of saved configurations.
This shows a wireframe for the generation output page that shows once the final column widths are submitted. Either a success message or error message will be shown, depending on the outcome of the PDF generation.
As a bonus, this shows a wireframe for a stretch goal of the project, which is to display a list of previous PDF generation jobs that the application has executed. Here, a download link to the ZIP file for a particular generation job will be available to users who want to download a previously created ZIP.
Front-end Implementation
Once the wireframes have been completed, the next step is to start visualizing the structure of the front-end and the layout of the Vue components in a component tree, shown below.
The component tree shows the parent and child components that will have data passed between them, either through props (from parent to child) or as arguments in custom event listeners (child to parent). At the highest-level is the App.vue component, which is the component that will contain all other components and is the component that is ultimately mounted onto the HTML page that is rendered in Document Object Model in the browser client-side. Inside App.vue, there are two main components: TheHeader.vue which will be the header with links shown at the top of every page in this application, and a router-view component which is a component from the Vue Router package that will dynamically render the page component linked to the route that is currently active.
In addition to these components which are imported into parent components and are therefore registered as local components, each of these components will also have access to globally registered components which are general user interface components, and can be used inside any Vue component without using an import statement.
BaseBadge is a component that can be used to wrap any piece of text inside a black container with rounded ends - this is mostly used to highlight column names and width percentages in the ConfigurationItems.vue component. Here, the column names and widths are inside a BaseBadge component.
BaseCard is a simple styled container that can be used by any component to wrap forms or list items.
BaseDialog is a pop-up window that will appear whenever an error is detected, as seen below. When the close button of BaseDialog or the gray backdrop surrounding the BaseDialog is clicked, the BaseDialog is closed.
BaseSpinner is a component that will display a loading animation inside its container when it appears. This can be used as a visual cue to indicate to the user that they should wait for an asynchronous process to finish.
The Vue front-end project will also contain a folder called the store, which is an area created by the VueX package for global state management. The store is where all components in the application can access global methods and data that affect what the user sees on-screen. The Store acts as a bridge between the front-end Vue components and the database. It is the part of the front-end where HTTP requests are actually made to the back-end and it is where the data fetched from the back-end is stored for access by front-end Vue components.
The root App.vue component will always display the TheHeader component, which is the dark blue banner seen at the top of every page in the application that contains two router-link components: ‘Inputs’ and ‘Configurations’. The Inputs component will initially display the ReportAndMapperInputs component below TheHeader when it is loaded to the document object model, as seen below. The default route ‘/’ that the user visits when the application is initially opened will also redirect to ‘/inputs’.
Clicking the “Benevity Report Generation Tool” header text in TheHeader will trigger the local method reloadHome(), which will either redirect the user to ‘/inputs’ if they are not on the ‘/inputs’ route or refresh the page if they are on ‘/inputs’.
Clicking ‘Configurations’ will direct the user to the ‘/configurations’ route using the Vue-router library, and visiting ‘/configurations’ will initially display the ConfigurationsList component below TheHeader, as seen in below.
The Inputs.vue component represents the page that will either display ReportAndMapperInputs (the first form), ColumnWidthPreview (the second form), or RequestComplete, depending on boolean variables that indicate which stage of the PDF generation the user is currently at.
When the user successfully completes the first form and clicks the ‘Next’ button, ReportAndMapperInputs will be hidden and the second form (the ColumnWidthPreview component) is displayed. If invalid inputs are entered, the forms will not switch and the user is shown which inputs are invalid.
The watermark image is optional. If no watermark image is uploaded, the default Benevity watermark image will be used for the generated PDF(s).
If the toggler prompting the user to enter a custom disclaimer text for the footer is unchecked, the default footer text will be used for the generated PDF(s).
Whether or not the input CSV file, banner image file, and watermark image files are File objects or strings depends on whether or not the ‘Load Configuration’ for an existing saved configuration was clicked in ConfigurationsList. If so, a hyperlink string containing the previously used file is now displayed in each of their respective inputs. Otherwise, the form will use the default empty values.
When the input CSV file is uploaded, the PapaParse JavaScript library is used to parse the column headers of the CSV file. Once this is complete, the mandatory column selection section appears below the PDF Report Inputs.
When a CSV is uploaded as a local file, the user will have the ability to choose whichever fields they want as these mandatory columns. However, when an existing saved configuration is loaded, the user will not be able to change the mandatory columns since the ColumnWidthsPreview component is already expecting a certain list of column names to display in the second form.
ColumnWidthPreview represents the second form that the user must fill before generating PDFs. There are three main sections in this component.
The column widths section is at the top of the component, where each adjustable column from the CSV is dynamically rendered as a table row containing a label, checkbox input, and text input for the width value of the column as a percentage. The checkbox enables the user to include or exclude the column from the table in the generated table.
The PDF preview panel section is a container that contains the ‘Generate PDF Preview’ button. If the total width, individual column widths, and number of columnWidths are valid. the loading spinner will appear and the panel will display an sample embedded PDF.
The last section is the configuration settings, which contain simple text inputs for the user to name their configuration, identify themselves as the creator, and write a brief description of what the configuration is.
The ‘Reset Form’ button can be clicked at any point to trigger a reset and display ReportAndMapperInputs (with default values) again while hiding ColumnWidthsPreview.
When the second form is successfully completed and the user clicks the ‘Submit’ button, ColumnWidthPreview is hidden and RequestComplete is displayed.
When the ‘Return to Initial Form’ button in RequestComplete is clicked, the boolean values are reset back to their original default settings and ReportsAndMapperInputs will appear once again.
RequestComplete.vue is a component that will indicate to the user that the PDF Generation is underway and that they will receive an email at the destination email address with a link to the ZIP file with the generated PDF(s) when it is generated. The user is also informed that the delivery time is dependent on the size of the file, and that their configuration will appear in the configurations list after the generation is complete.
ConfigurationsList represents the page that will display a list of ConfigurationItem components when the user clicks the ‘Configurations’ router-link from TheHeader component.
Each ConfigurationItem component will always display the configuration name, configuration ID, creation timestamp, creator name, and description. At the bottom of each ConfigurationItem are four action buttons:
One to show more details of the configuration (or hide them).
A ‘Download ZIP’ button that enables the user to directly download the generated ZIP file.
A ‘Load Configuration’ button that will redirect the user to ‘/inputs’ where ReportAndMapperInputs will have pre-populated values from the existing saved configuration.
A ‘Delete Configuration’ button that will delete the saved configuration with the matching ID from the back-end database.
If there are no saved configurations, a message “No configurations found.” will appear. There is also a ‘Refresh’ button that allows the user to send another request for saved configurations from the back-end.
The NotFound.vue component is a default page component that will appear when the user enters an invalid URL endpoint that is not ‘/’, ‘/inputs’, or ‘/configurations’.
Back-end Methodology
The back-end of the new PDF generator application was written in Java using Spring Boot framework. It consists of the following components:
A controller layer that handles HTTP requests from the Vue front-end application.
A service layer that interacts with the MySQL database and the AWS S3 bucket.
A repository layer that provides access to the data in the MySQL database.
A model layer that contains the entity classes used to courier data to and from the database.
The overall architecture of the back-end is designed to be scalable and reliable. The use of a MySQL database and an AWS S3 bucket will allow the application to handle a large number of users and PDF files.
Back-end Implementation
The design of the PDF generator application emulates the Model-View-Controller (MVC) design pattern. The view is the Vue front-end, which interacts with the controller by sending HTTP requests and displaying responses. The controller acts as the API between the front-end and the back-end. It routes requests to necessary service classes to implement the required business logic such as CSV parsing, email notification, PDF generation, and image processing. The service classes then uses model classes for data storage. Repository interfaces are used to store and access data in the MySQL database, while their corresponding entity classes act as data transfer objects. Below is a diagram of the back-end workflow.
Below is an Entity-Relationship Diagram of the database created in MySQL.
Below is a package diagram showing all Java classes and Vue components used.
Deployment
Our deployment strategy uses a modern DevOps approach to deploy our application stack onto AWS cloud infrastructure. We containerized Docker images in order to package frontend, backend, MySQL database, and all their dependencies. This ensures consistent deployment across environments. Our GitHub repository hosts our application code and configuration files. GitHub Actions will be utilized to automate a Continuous Integration and Continuous Deployment (CI/CD) pipeline, enabling us to build and test our code on every push. Once code passes these checks, the application is ready for deployment.
Terraform, an Infrastructure as Code (IaC) tool, defines our cloud resources and enables us to create the production environment in minutes by running a script. We model our AWS infrastructure, including the MySQL database, networking components, in Terraform configurations. The CI/CD pipeline will integrate with our Terraform configurations. When a pull request is merged to the main branch, GitHub Actions trigger the pipeline. The pipeline builds Docker images for the frontend and backend, tagging them with a version, and pushes them to a container registry. The Terraform configuration then deploys the updated infrastructure and the new Docker images. This workflow ensures synchronized updates to both the application code and infrastructure.
Finally, our deployment target is an AWS Elastic Compute Cloud (EC2), providing scalability, reliability, and manageability. AWS resources, such as S3 for storage, are integrated into our application and used to store the multimedia files.
Once the project was completed and the final presentation was complete, we also created a final report detailing the entire process and solution, and a set of user instructions for Benevity to follow so that they could deploy the application internally.
Outcome and Takeaways
Building this application was the culmination of all the key concepts we learned during Software Engineering Masters program at the University of Calgary. Out of all the projects I have worked on during my undergraduate studies, graduate studies, and industry work, this was by far the most difficult project, but also the most rewarding.
Our new application resulted in a 200% improvement in overall performance and functionality from the previous application it is replacing. We were able to meet all original requirements and the team earned an overall satisfaction score of 95% from the industry sponsor client, academic advisor, and instructor after the final presentation and demonstration, achieving a grade of A+.
Completing this project was a massive confidence boost for us, as we overcome many challenges and it affirmed that our decision to choose this project was the correct one. It gave each of us a better idea of which areas we needed address to become a better Software Developer. Personally, I feel better equipped to tackle any challenges that I encounter because of my experience with this project as I had never worked on a software application of this scale in a team setting.
Key technical challenges encountered during this project include:
Finding a solution that could work for the different number of column and column headers in each CSV.
Dealing with the styling limitations of the BFO PDF generator that was used.
Designing the database.
Managing and consolidating form data as the user progressed through the different forms in the front-end.
Dealing with the scenario in which a configuration is loaded with file hyperlinks from Amazon S3 instead of loading files locally. Built-in file input elements from the browser cannot contain pre-loaded values, so instead we built custom file inputs that could display both pre-loaded hyperlink values or locally uploaded file values.
When the PDF preview was generated, resizing the browser window would result the preview overlapping the edges of its container. This resulted in devising a creative solution where resizing the browser window would trigger a loading spinner, providing a buffer needed to reload a preview that is now resized to adjust for the new container size.
Key non-technical challenges encountered during this project include:
Time was a constant challenge throughout the project, as the condensed Spring and Summer schedule gave us less than three months (which was closer to 2.5 months since the first meeting with Benevity did not occur until the second week of May) to complete the project. Having to balance the Capstone workload with other classes during these semesters made the three months extremely fast-paced, with very little free time.
Learning curve: Because we did not have experience with any of the technologies listed by Benevity in the project description, the first month of the project was spent learning them. Learning the required skills and immediately figuring out how we can apply it in the context of our project was very challenging (but also very rewarded).
Communication: I had not worked with directly with the other two members in my team before, so it took time to acclimate to their work and communication styles. Additionally, we agreed in hindsight, that coding and collaborating in the same room was something we should have done earlier in the semester, as it became much easier to understand what each team member wanted from the other.
Possible Improvements Going Forward
The application can be further improved by following design guidelines from Skyline (Benevity's design system). Because functionality was our primary focus, using Skyline components became a lesser priority. If we had more time, we would also write automated unit and integrated unit tests in addition to all the exploratory manual testing we did.