Building a PhotoMesh Project Using an On-Premises Master and Storage

Before beginning a build using an on-premises master and storage, ensure that your fuser machines have access to your local storage, e.g., via AWS Direct Connect, Site-to-Site VPN, or Security Group and firewall rules. Sufficient bandwidth for the data transfer between your on-premises storage and the fuser machines is significantly affects production performance. See the overview to step 4 for some of the issues to consider regarding the hybrid architecture.

To use the configured environment to produce PhotoMesh projects using on-premises storage and a PhotoMesh that is installed on an on-premises computer while utilizing AWS fuser instances, follow the steps below:

1.     Make sure your on-premises computer has PhotoMesh installed.

Note:        File access can be achieved in different ways, e.g. via AWS Direct Connect, Site-to-Site VPN, or Security Group and firewall rules. Consult AWS documentation, support, your IT person, or Skyline support for more information.

2.     Create the directory: ".\PhotoMesh\AWSConfigurations", and copy the JSON files ("DataPrep_AT.json", "Reconstruction_CPU.json", and "Reconstruction_GPU.json") from the zip file downloaded in step 6b of "Customizing the (Fuser) Instance" to this directory.

3.     Start PhotoMesh and create a project.

Note:        The project must be created in a location that is accessible to all fuser instances.

4.     On the Home tab, in the Process group, click Build, then enter the Build Parameters and click Build.

5.     In the PhotoMesh Build Manger dialog, toggle on Use AWS Instances (Change).

6.     In the AWS Security dialog, enter your AWS credentials created in "Creating an IAM User for Running PhotoMesh Project Using an On-Premises Master".

7.     In the Auto-Scaling Fusers Settings dialog, do the following:

a.     Check the Use AWS Instances check box.

b.     Maximum Price Per Spot Instance: Set the maximum price you want to pay per EC2 spot instance. For more information see Amazon EC2 Spot Instance Pricing in Amazon’s AWS documentation.

c.     Maximum Instances: Set the number based on the number of EC2 instances you want to run.

d.     Launch Rate: Set the maximum number of fuser instances to launch per minute, e.g. if Maximum Instances was set to 200, and the Launch Rate to 50, the Build Manager will spread the launch of the fusers over 4 minutes. This prevents a potential bottleneck that could be caused by a large number of fusers attempting to read the build data at the same time.

e.     Terminate AWS instances when build finished: Select the check box if you want to terminate AWS fuser instances when the build is complete.

f.      Build Priority: Select either Choose all listed options, to use all machine types listed in the JSON file, with priority based on the order they are listed in the file, or select a specific machine.

g.     Security Group ID: Select PM_SG.

h.     Subnet ID: Click Additional Subnets... and type a list of all the subnet ID's you want to use. These were created in "Configuring a VPC". A subnet must be created for each availability zone.  See the overview to step 4 for a comparison of onDemand/spot instances and information on using pricing history to determine instance availability.

i.      Key Pair Name: Select PM_KeyPair.

j.      Configuration Name: Select based on the current processing step. Select the Reconstruction_GPU option if you want to run an entire build without needing to reselect the configuration before each build step, since this option can be used for all build steps.

·         Reconstruction_GPU – For texture and output steps.

·         AT_MultiCoreCPU – Multi-core CPU for data preparation and aerotriangulation steps.

·         Reconstruction_CPU – For point cloud and model steps.

Note:    These configuration options are derived from the JSON files copied in step 2 of "Building a PhotoMesh Project Using the Master Instance". Each of these JSON files contains an array of instance types that PhotoMesh should use, ordered based on preferability. During production, PhotoMesh iterates over the elements of the array, beginning with the first instance type, until it finds an available instance.
Each instance object in the array has four properties: ami, instanceType, demandType, and fusersCount. The instanceType defines the hardware of the host computer used for your instance. Each instance type offers different compute, memory, and storage capabilities. The demandType determines the pricing arrangement: onDemand or spot. The fusersCount determines the number of PhotoMesh processes to run on each instance. See the overview to step 4 for some of the issues to consider when selecting instance types.

k.     Click Edit (next to Configuration Name), insert in all fields the AMI ID of the PM Linux Fuser Image (created in "Saving a Fuser AMI"), and then click OK.

l.      Select Shutdown inactive fuser instances after and set its value to "1".

m.   Click Edit Startup Script and paste the template script from the zip file downloaded in step 6b of "Customizing the (Fuser) Instance (Startup_Script_Template.txt). Edit the script so it includes the username and the password of your PM Master machine set in step 13 of "Connecting to the (Master) Instance" (In the script, replace all <serverip >, <share>, <pass> and <user> placeholders with the required value).

n.     Click OK.

8.     Click Build.

Note:        A message is displayed informing you that no available fusers are currently running and asking if you want to start a local fuser. If you want to start a fuser on the master machine to take advantage of its computing resources, click Yes. If you are running many spot fuser instances, however, it is recommended to click No to free up computing resources for management of the build and file server tasks.

Note:        If a message is displayed that "PhotoMesh’s fuser request was rejected by AWS, since it exceeded the instance limit", you need to request a limit increase for "Spot instance requests". See: Amazon EC2 Service Limits in Amazon’s AWS documentation for more information.

9.     In the PhotoMesh Build Manger dialog, you can monitor the creation and automatic use of the automatically launched fuser instances. You can also monitor the creation of your instances by choosing Instances in the navigation pane of the Amazon EC2 console, and reviewing the custom Tags added to PhotoMesh launched instances: Build start time, Owner, Project, Type, and Working folder.

10.   It is recommended to verify that the instances are terminated according to the settings defined in the AWS Cloud Settings dialog, particularly the first time they are terminated.