Today I tried to set up my jigsaw-photo-stream project to use netlify large media with git-lfs.

What I failed to read was this key part of the netlify documentation:

Files tracked with Large Media are uploaded directly to the Netlify Large Media storage service on push, completely bypassing the site build.


As part of the build process for the project I resize the images that are uploaded.
The netlify documentation suggests that you could use netlify’s image transformation to get around this but ideally I dont want the project to be tied into the service provider should I want to migrate to another static site host.

At this point I’d already set up git-lfs on the repo and my netlify builds were failing.
I found this link to remove git-lfs from an existing repo

The steps being:

1
git lfs uninstall

remove lfs stuff from .gitattributes

1
2
3
4
5
6
7
8
git lfs ls-files | sed -r 's/^.{13}//' > files.txt
while read line; do git rm --cached "$line"; done < files.txt
while read line; do git add "$line"; done < files.txt
git add .gitattributes
git commit -m "unlfs"
git push origin
git lfs ls-files
rm -rf .git/lfs

Comment and share

Today I was working on an API written in Laravel for a React Native app with another developer.

He was trying to make requests to the Laravel backend and told me he kept receiving a response with a http status code of 302. 3XX http status codes are redirection status codes.

It turned out that he had not set an accept header on the requests to the server that the app was making.

By default if you dont set a requests accept headers they default to Accept: */*. With those set laravel responds with a Content-Type of text/html.

For the purpose of this project every api response needs to return JSON.

We can achieve this really easily by using middleware to overwrite the Accept headers on the incoming request and setting them to application/json.

Continue reading

At our company setting up gitlab ci configuration is one of the jobs I end up doing by default.

This weekend I wrote a package to help speed that process up by generating a .gitlab-ci.yml file as well as installing some of the packages and configuration files to make the following possible:

The package currently provides a single artisan command to do all of the above after answering a few simple questions.

Check the repo out here:

https://github.com/talvbansal/laravel-gitlab-ci-config-generator

Continue reading

Over the last couple of days of social distancing I spent some time working on a photostream site for some of my travel photos.

Whilst I usually post them on mine and my wife’s Instagram page iwantthewindowseat I’ve not found myself having the motivation to select an image, think of a caption, find hashtags and post at the best time for visibility as much as I used to.

I also wanted somewhere where copyright ownership wasn’t an issue. Much like this site is an archive of my ramblings and things I’ve worked on, I thought it would be cool to hav something similar for my photos.

This project was also a great opportunity to look at some of the newer browser features like lazy loading and leverage them - as of writing Firefox 74 is out and native lazy loading is due in Firefox 75. Native lazy loading is in chrome, the project uses a polyfill to bring lazy loading to older browsers.

Photo Stream

The project itself can be seen at iwantthewindowseat.netlify.com.

The code repository can be forked and cloned over here.

Comment and share

Recently I wrote about my current Gitlab CI process, when it came to the deployment part of the process I showed how I was handling it using a tool called Laravel deployer but I didn’t breakdown what laravel deployer was doing and how I had it configured.

The Laravel deployer docs are pretty good however I found a couple of server config issues that I always find myself referring back to when setting up auto deployment. Mostly to automatically restart Laravel Horizon and restarting Php-fpm without needing sudo privileges.

Lets imagine I was going to a set up Gitlab CI / CD for a fictional project hosted over on the fictional domain of deployer.talvbansal.com with a real repository here that is hosted on somewhere like Linode, Digital Ocean or even AWS.

Continue reading

Introduction

My most read articles on this blog are about Gitlab CI/CD with PHP. They cover a basic linting, testing and crude deploying process.

Today I want to look at my current CI/CD process for my Laravel projects in more depth. Currently the pipelines of my projects might vary slightly but is very similar to this:

Gitlab Pipeline

So there are 5 main stages in the process:

  • Preparation - The pulling down of dependencies and storing them in an artifact
  • Syntax - Check code syntax
  • Testing - Run unit tests
  • Building - Build assets
  • Deployment - Deploying to an appropriate server

The stages are processed in order with each stage containing one or many tasks. Should one task fails in a stage then the whole pipeline stops and is marked as failed.

To run the pipelines I make use of gitlabs free tier which gives you access to 2000 shared minutes per month as well as a runner on a server I have. More about setting that up can be found here.

Continue reading

I recently watched the following great talk on hacking laravel apps.

Towards the end of the talk Antti shows how it is possible to potentially gain root access to a server if your scheduler is running as root too.

As soon as I saw it I know I had a couple of apps where this vulnerability could have been exploited and so went to patch them straight away.

Whilst I knew what needed to be done I wasn’t 100% on how exactly I’d add an entry into another user’s crontab that wasn’t my own or root.

Turns out it was quite simple, acting as root use the -u argument to specify the target user.

1
sudo crontab -e -u www-data

In the above example the crontab for the user www-data would be opened. Since my php-fpm instance is run by www-data and therefore has access to all the application code already this made sense to me.

Hopefully I’ll never make this mistake again. If you haven’t already seen Antti’s talk above I’d highly recommend doing so asap!

Continue reading

For one of my recent projects I wanted to make use of the free allowance that AWS gives for SES.

One of the conditions of the SES allowance was that your calling app needs to be hosted on EC2.

I’ve not used EC2 before so I figured this would be a good way to dive into it.

Whilst I would never usually install software like composer on a production server, this was purely to test things out.

So after signing up for AWS and creating a local ubuntu server on an EC2 t2micro instance then cloning down the project I ran composer install to come across the following message:

1
2
3
4
5
6
7
8
9
10
composer install
Loading composer repositories with package information
Updating dependencies (including require-dev)

mmap() failed: [12] Cannot allocate memory

mmap() failed: [12] Cannot allocate memory
PHP Fatal error: Out of memory (allocated 822091776) (tried to allocate 4096 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/Solver.php on line 223

Fatal error: Out of memory (allocated 822091776) (tried to allocate 4096 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/Solver.php on line 223

822091776bytes is over 800mb of memory being consumed by composer.

Continue reading

Introduction

I recently build my first site with Jigsaw and deployed it on Netlify.

As part of that project I had to get some data from the Instagram API and present a feed of the latest 5 images on the site.

Rather than dealing with CORS errors in javascript I wondered what I could get away with in PHP during the build phase on a statically generated site.

Would I be able to:

  • Query some form of public json endpoint
  • Find the urls for the latest 5 images
  • Download them locally resize them for efficiency
  • Display them using Jigsaw
Continue reading

Introduction

Laravel apps read sensitive information from their .env file.

Recently I found out that Laravel Mix can pass values from the same .env file to the js portion of your app as long as they are prefixed with MIX_

I use Gitlab ci pipelines to build production assets so that I dont need that additional tooling on the production servers the main one being:

  • yarn run production

This is preceded with cp .env.example .env meaning when the build commands are being run, they are going to use values from the .example.env file.

If your project doesn’t make use of anything from the .env file then this is totally fine, however in scenarios where you do, since production applications will almost certainly have different .env values to those in the .example.env file (Never commit credentials to source control!) the resulting file will have been built with the wrong credentials.

In this article I’m going to show how you can use gitlab CI to build those assets with updated environmental variables so that they function as expected when deployed to your production servers.

Continue reading
Author's picture

Talv Bansal

Full Stack Developer, Part Time Photographer


Head of Software Engineering


Remote