HTML5 Curation: Difference between revisions

From Flashpoint Datahub
Jump to navigation Jump to search
(Placeholder)
 
m (→‎Missing assets: Grammar fixes.)
(31 intermediate revisions by 7 users not shown)
Line 1: Line 1:
In this guide I’ll explain how to curate HTML5 games. This will only cover games that don’t require a persistent server connection.
HTML5 games are not nearly as endangered as Flash games, and are in fact often cited as replacing the Flash medium. However, they still pose a risk of disappearing at some point in time, and therefore should be considered for curation just like other web game formats.


== Recognizing an HTML5 game ==
The following page is a tutorial on how to curate HTML5 games to add to Flashpoint.


Without looking at a web page’s source, there’s no definitive way to tell if you’re dealing with an HTML5 game. You can however easily discard Flash by whether a lego brick-shaped plug-in icon is displayed to the left of the address bar. The same goes for any technologies that make use of browser plugins (e.g. Java).
== Before You Start ==


[[File:PluginIcon.PNG|plugin]]
This tutorial will assume you've read the standard [[Curation Tutorial]], which is highly recommended to go through. Try curating a Flash game before moving on to HTML5.


== Required software ==
== Identifying HTML5 Games ==


These will be required to perform the curation process:
If a web game doesn't ask to enable Adobe Flash Player and there's no sign or mention of any of the other [[Platforms]], it is very likely an HTML5 game. Some use game engines as their base, the most prominent example being UnityWebGL, which replaced the previous [[Unity_Curation|Unity NPAPI implementation]] that no longer works.


* [https://bluemaxima.org/flashpoint/downloads/ Flashpoint Core] (Optional, recommended)
To confirm this, press F12 to open the Developer Tools of your browser, click on the Inspect Element button and click on the game area. This will show you in the <code>Elements</code> tab the iframe the game is located in, and there you'll find the link for the main file of the game. You can also look for a <code>.html</code> file, likely with the filename <code>index.html</code>.  
* [https://eternallybored.org/misc/wget/ wget for Windows]


The guide below will explain how to use them.
Copy the file's link and go to it. If it plays an unobstructed version of the game, you've found the source of your Launch Command. Keep in mind that not all games should be divorced from the page they are found in, such as individual games outside of webgame websites.


== Curation ==
'''Make sure to always use http:// instead of https:// to make sure Flashpoint grabs files from the fake Internet instead of the real one!'''


In order to curate an HTML5 game, we first want to retrieve all the URLs required by the specific game we’re dealing with.
== Grabbing The Game From The Internet ==


There’s two ways to do so:
[[File:Individual-Sprites-HTML5-example.png|thumb|300px|Some games have one or more image files that act as sprite sheets, others will have each unique texture as an individual file, so make sure to get them all.]]


* Using a web debugging proxy like Fiddler or Charles Proxy
There are an extremely low number of HTML5 games that are single-asset. Your curation is almost guaranteed to be multi-asset. From here, you have two main options, both of which are [[Curation Tutorial#Multi-Asset_Games|described here]], but each is more suited for different tasks which we'll explain here.
* Using the browser’s console


I personally prefer using Charles Proxy, I also use it for curating Flash games, which requests are not displayed in the browser’s console. I like its interface and it allows you to view the directory hierarchy of all HTTP requests.
Create a new curation in the Curate tab of Flashpoint Core and fill it out with as much information as you can, including the Launch Command you found earlier. The Platform should be <code>HTML5</code>, and the Application Path should be <code>FPSoftware\Basilisk-Portable\Basilisk-Portable.exe</code>, referring to the Basilisk browser used for HTML5 games.


For the purposes of this guide I’m going to keep it simple and stick to Firefox’s console, which should be similar enough to that of Chrome. If you do opt for a debugging proxy it’s imperative to clear the browser’s cache EVERY time you attempt to capture the game’s requests.
After inputting the launch command, use <code>Run with MAD4FP</code>, and start playing the game. [[MAD4FP]] will download any assets automatically as you play and make progress.


I’ll illustrate each step with an example of a real HTML5 game. I’m going to attempt to curate Google’s snake clone that you get by performing a search for “snake”.
'''REMEMBER, ALWAYS TEST THE GAME 100%, NO MATTER HOW BIG OR SMALL!''' This is true for all games, but especially HTML5, as nearly none are single-asset, you are almost guaranteed to find a multi-asset. Because of how it works, most games will only call assets when needed, so make sure to navigate all the menus, click all the buttons, and play the game as thoroughly as possible. This may include hidden collectables, unique voice lines, downloadables, completion bonuses, etc.


Here’s our prey:
Wayback will not save these missing assets on a normal capture, and computers/browsers tend to clear cache much more frequently for optimization, making cache dumps unlikely to yield anything, so once it's gone, it's likely gone for good. Better to catch everything first try and make sure it's not lost. for more information on multi-assets in general, [[Curation Tutorial#Multi-Asset_Games|go here]].


[[File:Snake.png|screenshot]]
After making sure you got the whole game, test it again, this time without using MAD4FP pressing the <code>Run</code> button.


Let’s start off by opening the console, we can easily do this by pressing '''Ctrl+Shift+I''' or by right clicking anywhere in the page and then selecting “Inspect”. Then navigate to the “Network” tab.
== Responsive Design Mode and Mobile Elements ==
[[File:DesktopvsMobileHTML5.png|thumb|200px|Some games will have mobile controls, loading different assets.]]


If any requests are already shown, click on the “Clear” icon to get a fresh start. While this tab is selected, our browser will record all requests made by the page and display them as a list.
Unlike traditional plugins, HTML5 allows games to be played on both Desktop and Mobile. This means that some games will implement Responsive Design elements so games work better on mobile platforms. This can load new assets such as the pages, scripts, elements, images, etc.


Refresh the page, this will cause it to make a bunch of requests to display the page. As the game loads, a bunch more requests will be made.
To check whether a game has these elements, play the game both in desktop and Responsive mode. On Basilisk in the developer console, enable <code>Responsive Design Mode</code>, then on the Responsive menu enable <code>Touch Events</code>, and make sure to reload the page as these elements might not have been loaded on Desktop mode. Use the <code>Rotate</code> button to test, as some games do not allow you to play in a certain orientation and will load warning messages.


Now, here’s where it gets tricky because most HTML5 games won’t request all of the files it needs at once but rather will request them sort of on-demand.
Make sure to skim through the entire game where these elements might show up, like parts with game controls or menus.


Imagine a game that has multiple stages, and stage two uses a different music track, it may not request it until that stage starts. Depending on the game though, it is sometimes possible to retrieve a list of most if not all of the files it needs by exploring the contents of other files it requests.
== Troubleshooting ==


This could be an XML or JSON file that lists the music track to be used for each stage, for example. However, I won’t get into detail for now. This game does not require any of that.
=== https removal ===


Play around with the game until no more requests are made or you feel satisfied. Then look for a button named “HAR”, it should be in the top right of the editor in Firefox, download icon in Chrome. Click it and select “Save everything as HAR”.
Don't be surprised if it doesn't work at first, HTML5 games are relatively stable and mostly work, but a common occurrence is the game refusing to load assets.  


This file contains the data of all the requests listed in the console. You can quickly extract all the requested URLs by opening the file in a text editor (i.e. notepad) and copy/pasting the contents in the left text box shown [https://jsfiddle.net/XXLuigiMario/pxcmew9t/show here].
This most likely is because of <code>https</code> requests inside the code of the game calling for files on the live web and failing, instead of <code>http</code> which calls local files. You will need to find and replace all instances of <code>https</code> with <code>http</code> in the game files. You might need to repeat this step multiple times for different parts of the game, due to most games being muiltiassets, they will only call certain parts that are needed during sections of gameplay.  


Copy the URLs from the right and save them in a new text file in its own dedicated folder.
To streamline and simplify the proccess, we suggest using Notepad++ and its <code>Find in Files</code> function, where you can input the Directory that points to the content folder of your curation, and then click <code>Replace in Files</code> where it will search in all the game files each instance and replace it.


At this point, make sure you have a copy of Flashpoint for testing. Flashpoint Core is recommended but any version can be used.
This is a really common form of modification for HTML5 games, but it is minor enough that games with altered https requests don't really need to be marked as Hacked. Note that this rule only applies to text files (such as HTML and JS files); you must always mark a game as Hacked if you edit binary files (such as SWFs).


Next up, we’ll need to use a tool called wget. Chances are you have heard about it before, you can download it for Windows [https://eternallybored.org/misc/wget/ here].
=== Missing assets ===
While curating, there might be a few missing files, so to get them we can use cURLsDownloader;


Save it in the same folder as the text file with the URLs, then open a command line window. To do so, simply press '''Win+R''', type “cmd” and then '''Enter'''.
* If MAD4FP isn't able to download certain files, or they are missing while testing without it, go to the Logs tab, click on <code>Copy 404 URLs</code> , create a <code>.txt</code> file, paste and save.
* If Basilisk is having trouble running the application, run the game on the live web with the network tab open, and play the game as if you were curating it as normal. After finishing, copy all cURLs, create a <code>.txt</code> file, paste and save.  


Type “cd”, without quotes, followed by the path to the folder where you saved your text file and '''Enter'''. Example: cd C:\Users\Foo\Game\
Next, drag & drop the file onto <code>cURLsDownloader.bat</code>. Copy the contents of the <code>Downloads</code> folder outputted by cURLsDownloader, select the <code>Open Folder</code> button on your curation in the Curate tab, and paste the folder into the <code>content</code> folder of your curation.


Then enter “wget -x -i filename.txt” and let it do its thing. Once all files have been downloaded, copy the resulting folder(s) to the “htdocs” folder in your Flashpoint directory, you will find it under a folder named “Server”.
=== Clean-up of junk Files ===


Navigate into a folder named “FPSoftware” next to where you found “Server”, find an executable for a web browser named Basilisk in it and run it. Copy the base URL of the game into its address bar and press '''Enter'''.
Since MAD4FP downloads every file requested by the game that still exists on the live web, you can get a lot of files that are unnecessary for the game to work. Be careful not to enter pages outside of the game as it will download them too, best way to prevent this is by knowing where this happens inside the game.


If the game works then you’re almost done! :)
Even with these precautions, you will download junk from other services the website uses outside of the game. After deleting them, make sure the games still work without them.
 
=== Broken or non-functional games ===
 
Some games might require a certain browser we do not support or newer features not available on the 32bit version of Basilisk, which will no longer receive updates. This version is used to ensure compatibility with 32bit machines, and also because it houses most NPAPI plugins. Staff is working on implementing a replacement browser for these newer HTML5 titles. Make sure to ask staff if this is really the problem.
 
If yes, although unable to be played on the current version of Flashpoint being used, it should still be saved, as these features are likely to be supported in the future and the game work as a result.
 
For this situation, download the [[Launcher Extensions|Browser Mode Launcher Extension]], and type <code>:browser-mode:</code> in the application path field. This will use Flashpoint's built in Browser, which is more feature complete. This will fix most compatibility issues, you can still use MAD4FP with it.
 
Make sure to write a note explaining that the game does not currently run in flashpoint:
<pre>Game only currently works with the :browser-mode: application path extension as of Flashpoint version X.X, please test once a new version is released and remove this sentence if it's able to work without it.
</pre>
 
If it still doesn't work, the best option is to run the game on the live web with the network tab open, and playing the game as if you were curating it as normal. After finishing, copy all cURLs, create a <code>.txt</code> file, paste and save. Next, drag & drop the file onto <code>cURLsDownloader.bat</code>. Copy the contents of the <code>Downloads</code> folder outputted by cURLsDownloader, select the <code>Open Folder</code> button on your curation in the Curate tab, and paste the folder into the <code>content</code> folder of your curation.
 
Upload this curation to #curator-lounge on our Discord server for safekeeping, as it might be able to work in the future.
 
== Wrapping It Up ==
 
Follow the proper [[Curation Format]], just like with any curation.
 
Click on <code>Export</code> button at the bottom of your Curation, this will create a single <code>.7z</code> file containing the entire curation, which can be found by clicking <code>Exported Curations</code> on the Curate tab, which opens the folder. Once you have the file, submit it to the #other-game-curations channel of the Flashpoint Discord, typing the name of the game and using the HTML5 emoji when submitting. If the file is too large for discord, upload it to https://other.curations.unstable.life/, and type the name with the emoji in the Discord channel.

Revision as of 11:57, 19 January 2021

HTML5 games are not nearly as endangered as Flash games, and are in fact often cited as replacing the Flash medium. However, they still pose a risk of disappearing at some point in time, and therefore should be considered for curation just like other web game formats.

The following page is a tutorial on how to curate HTML5 games to add to Flashpoint.

Before You Start

This tutorial will assume you've read the standard Curation Tutorial, which is highly recommended to go through. Try curating a Flash game before moving on to HTML5.

Identifying HTML5 Games

If a web game doesn't ask to enable Adobe Flash Player and there's no sign or mention of any of the other Platforms, it is very likely an HTML5 game. Some use game engines as their base, the most prominent example being UnityWebGL, which replaced the previous Unity NPAPI implementation that no longer works.

To confirm this, press F12 to open the Developer Tools of your browser, click on the Inspect Element button and click on the game area. This will show you in the Elements tab the iframe the game is located in, and there you'll find the link for the main file of the game. You can also look for a .html file, likely with the filename index.html.

Copy the file's link and go to it. If it plays an unobstructed version of the game, you've found the source of your Launch Command. Keep in mind that not all games should be divorced from the page they are found in, such as individual games outside of webgame websites.

Make sure to always use http:// instead of https:// to make sure Flashpoint grabs files from the fake Internet instead of the real one!

Grabbing The Game From The Internet

Some games have one or more image files that act as sprite sheets, others will have each unique texture as an individual file, so make sure to get them all.

There are an extremely low number of HTML5 games that are single-asset. Your curation is almost guaranteed to be multi-asset. From here, you have two main options, both of which are described here, but each is more suited for different tasks which we'll explain here.

Create a new curation in the Curate tab of Flashpoint Core and fill it out with as much information as you can, including the Launch Command you found earlier. The Platform should be HTML5, and the Application Path should be FPSoftware\Basilisk-Portable\Basilisk-Portable.exe, referring to the Basilisk browser used for HTML5 games.

After inputting the launch command, use Run with MAD4FP, and start playing the game. MAD4FP will download any assets automatically as you play and make progress.

REMEMBER, ALWAYS TEST THE GAME 100%, NO MATTER HOW BIG OR SMALL! This is true for all games, but especially HTML5, as nearly none are single-asset, you are almost guaranteed to find a multi-asset. Because of how it works, most games will only call assets when needed, so make sure to navigate all the menus, click all the buttons, and play the game as thoroughly as possible. This may include hidden collectables, unique voice lines, downloadables, completion bonuses, etc.

Wayback will not save these missing assets on a normal capture, and computers/browsers tend to clear cache much more frequently for optimization, making cache dumps unlikely to yield anything, so once it's gone, it's likely gone for good. Better to catch everything first try and make sure it's not lost. for more information on multi-assets in general, go here.

After making sure you got the whole game, test it again, this time without using MAD4FP pressing the Run button.

Responsive Design Mode and Mobile Elements

Some games will have mobile controls, loading different assets.

Unlike traditional plugins, HTML5 allows games to be played on both Desktop and Mobile. This means that some games will implement Responsive Design elements so games work better on mobile platforms. This can load new assets such as the pages, scripts, elements, images, etc.

To check whether a game has these elements, play the game both in desktop and Responsive mode. On Basilisk in the developer console, enable Responsive Design Mode, then on the Responsive menu enable Touch Events, and make sure to reload the page as these elements might not have been loaded on Desktop mode. Use the Rotate button to test, as some games do not allow you to play in a certain orientation and will load warning messages.

Make sure to skim through the entire game where these elements might show up, like parts with game controls or menus.

Troubleshooting

https removal

Don't be surprised if it doesn't work at first, HTML5 games are relatively stable and mostly work, but a common occurrence is the game refusing to load assets.

This most likely is because of https requests inside the code of the game calling for files on the live web and failing, instead of http which calls local files. You will need to find and replace all instances of https with http in the game files. You might need to repeat this step multiple times for different parts of the game, due to most games being muiltiassets, they will only call certain parts that are needed during sections of gameplay.

To streamline and simplify the proccess, we suggest using Notepad++ and its Find in Files function, where you can input the Directory that points to the content folder of your curation, and then click Replace in Files where it will search in all the game files each instance and replace it.

This is a really common form of modification for HTML5 games, but it is minor enough that games with altered https requests don't really need to be marked as Hacked. Note that this rule only applies to text files (such as HTML and JS files); you must always mark a game as Hacked if you edit binary files (such as SWFs).

Missing assets

While curating, there might be a few missing files, so to get them we can use cURLsDownloader;

  • If MAD4FP isn't able to download certain files, or they are missing while testing without it, go to the Logs tab, click on Copy 404 URLs , create a .txt file, paste and save.
  • If Basilisk is having trouble running the application, run the game on the live web with the network tab open, and play the game as if you were curating it as normal. After finishing, copy all cURLs, create a .txt file, paste and save.

Next, drag & drop the file onto cURLsDownloader.bat. Copy the contents of the Downloads folder outputted by cURLsDownloader, select the Open Folder button on your curation in the Curate tab, and paste the folder into the content folder of your curation.

Clean-up of junk Files

Since MAD4FP downloads every file requested by the game that still exists on the live web, you can get a lot of files that are unnecessary for the game to work. Be careful not to enter pages outside of the game as it will download them too, best way to prevent this is by knowing where this happens inside the game.

Even with these precautions, you will download junk from other services the website uses outside of the game. After deleting them, make sure the games still work without them.

Broken or non-functional games

Some games might require a certain browser we do not support or newer features not available on the 32bit version of Basilisk, which will no longer receive updates. This version is used to ensure compatibility with 32bit machines, and also because it houses most NPAPI plugins. Staff is working on implementing a replacement browser for these newer HTML5 titles. Make sure to ask staff if this is really the problem.

If yes, although unable to be played on the current version of Flashpoint being used, it should still be saved, as these features are likely to be supported in the future and the game work as a result.

For this situation, download the Browser Mode Launcher Extension, and type :browser-mode: in the application path field. This will use Flashpoint's built in Browser, which is more feature complete. This will fix most compatibility issues, you can still use MAD4FP with it.

Make sure to write a note explaining that the game does not currently run in flashpoint: 

Game only currently works with the :browser-mode: application path extension as of Flashpoint version X.X, please test once a new version is released and remove this sentence if it's able to work without it.

If it still doesn't work, the best option is to run the game on the live web with the network tab open, and playing the game as if you were curating it as normal. After finishing, copy all cURLs, create a .txt file, paste and save. Next, drag & drop the file onto cURLsDownloader.bat. Copy the contents of the Downloads folder outputted by cURLsDownloader, select the Open Folder button on your curation in the Curate tab, and paste the folder into the content folder of your curation.

Upload this curation to #curator-lounge on our Discord server for safekeeping, as it might be able to work in the future.

Wrapping It Up

Follow the proper Curation Format, just like with any curation.

Click on Export button at the bottom of your Curation, this will create a single .7z file containing the entire curation, which can be found by clicking Exported Curations on the Curate tab, which opens the folder. Once you have the file, submit it to the #other-game-curations channel of the Flashpoint Discord, typing the name of the game and using the HTML5 emoji when submitting. If the file is too large for discord, upload it to https://other.curations.unstable.life/, and type the name with the emoji in the Discord channel.

Retrieved from "https://flashpointarchive.org/w/index.php?title=HTML5_Curation&oldid=4267"