SDL MVP retreat 2017

MVPs are few chosen Tridion professional who are selected based on yearly contribution to SDL community. SDL pampers those professional with work, sea food, music and unlimited Daaru in retreat. SDL retreat provides chance to visit Portugal again if you miss any place during last years.

SDL retreat is one of the most awaited event of the year for MVPs. It motivates for next year.

"This is always 5 days event, SDL treats you like Damad except few Fatwas:

• From the moment you arrive to your departure airport until you fly back home, nearly all expenses are covered. If you decide to get drunk in a bar at the airport, I expect that you earn enough to pay for it yourself and not present me with an expense receipt for it, or I’ll remember next year.
•  If you lose your flight due to above mentioned state of drunkenness, I also expect you to be man enough to realize it’s your damn problem to fix.

Few tips from one of the Permanent and Veteran Member

1. Keep on the right side of Carla
2. Don't try to out-drink Nuno 
3. Don't expect to go home thinner
4. If you at all musical (even slightly), we will expect you to enztertain us, and yourself. Even long-buried musical talents have been unearthed at the retreat. But there are no rules. And of course, bring your instrument(s).
5. Expect the unexpected from your fellow-MVPs and yourself

My Biggest worry:

"What kind of Veggy food, No eggs" will be served to me in. *T*: Veggy food in Portugal?

Few Testimonials 

Carla:

"Well I'm actually thinking of doing all vegetarian meals FOR EVERYBODY and alcohol free retreats from now on"

Curlette:

"Great to have you back and finally someone more picky than me with food :) Already making me look better Haha ;)"

But to my surprise food was really good this time with few exceptions. My biggest helper was Garlic butter their which provided taste to food.

Day 1:

People gathers at Airport to meet and greet, then drivers drove to the Hotel Pousada de Ourem in beautiful city of Ourem. People drinks, eats and sleep to get early for the big day.

Day 2:

MVPs gathers at conference room for connect on upcoming changes in SDL product and road-map. Nuno and other Sdl teams showcase ongoing work. Some goodies to collect on EOD.

Day 3 & 4

Time to code and shocase next day. I teamed up with The Nuno and Sayantan to develop "Poor Man's AWS Cloudwatch Monitor" We successfully created & scheduled it on  Windows Scheduler to send queue status every 2-3 minutes to cloud watch. Night is full of fun, drinks and special musical performances by Quirijn, Raimond on Piano, Guitar..  give them any Instrument, these guys are exceptionally good. Rick and Johnathan also surprised with their Piano and singing skills respectively. Dominic also tried something with his mouth instruments before sleeping. 

also we saw some nazare @  Nazaré

Day 5

Time to say Good Bye & promises to meet again.

Publishing queue status on AWS Cloud-watch

In this years MVP retreat we did small Proof of concept to show the Publishing queue status on AWS cloud-watch in nice format. There are various ways to accomplish it.

- Using CoreService API to access the Queue.
- Directly querying the DB (not very recommended)
- PowerShell to read the queue (Yes its possible)

We choose last option as some Tridion Powershell Module created by SDL MVP Pkjaer makes it very easy.

Steps:

- Install Tridion Powershell module, details in above link.
- Install AWS Powershell module(s) on machine to work.
- Here is code snippet which do following

• Import AWS Powershell 
• Create Cloudwatch matrices based on two states "Waiting for Publish" and Waiting for Deployment"
• fill the matrices with count of above two states
• Write the data to cloudwatch

 Import-Module "C:\Program Files (x86)\AWS Tools\PowerShell\AWSPowerShell\AWSPowerShell.psd1"  
 $dat = New-Object Amazon.CloudWatch.Model.MetricDatum  
 $dat.Timestamp = (Get-Date).ToUniversalTime()  
 $dat.MetricName = "Waiting For Publish"  
 $dat.Unit = "Count"  
 $dat.Value = (Get-TridionPublishTransaction | Where-Object {$_.State -eq "WaitingForPublish"}).Count  
 $dat1 = New-Object Amazon.CloudWatch.Model.MetricDatum  
 $dat1.Timestamp = (Get-Date).ToUniversalTime()  
 $dat1.MetricName = "Waiting For Deployment"  
 $dat1.Unit = "Count"  
 $dat1.Value = (Get-TridionPublishTransaction | Where-Object {$_.State -eq "WaitingForDeployment"}).Count  
 set-AWSCredentials -AccessKey -SecretKey FT1nwN74NHkGtlJ -StoreAs "MVP17"  
 Set-AWSCredentials -ProfileName "MVP17"  
 Write-CWMetricData -Namespace "Usage Metrics" -MetricData $dat  
 Write-CWMetricData -Namespace "Usage Metrics" -MetricData $dat1  

Auto-scaling:

One of the purpose to push the queue data to AWS Cloudwatch is to configure the auto-scaling of instances.

if there are lots of items in queue, auto-scaling of publishing system can be configured on AWS infrastructure.

Migrate Images in Media Manager automation

Consider the use-case of uploading 7500 images manually and linking each image with component in SDL  Tridion. Its very tedious tasks to do and if any change in images and need to redo 2-3 times. you need hell of a time and dedication.

So Best approach is to Automate set of tasks so that it can be imported into Media Manager and Linking can also happen side by side.

So need following set of services to accomplish this task:

1. SDL Tridion
2. Media Manager Cloud instance
3. STS Service
4. SDL Media manager Client
5. ECL Service Client

Part 1

Connecting to Media manager

STS - Security token service (STS) is a cross-platform open standard core component of the OASIS group's WS-Trust web services single sign-on specs.

STS is used to authenticate between Migration Tool and SDL media manager. setting up it required configuring public key of the  Media manager. It was the hardest part for this whole stuff.

- Generate proxy from the Media Manager Service
-  STS service should be in the running state.
- Migration tool will connect to MM using Federated Security with STS service providing the encoded MM's public certificate.
- Once connection is established the upload process triggers.

There would be two bindings in app.config  one for connecting to mediaManager another to connect to STS service running on Url like following : http://localhost:89/IWSTrust13

Note: STS service is not provide OOTB by SDL. You have to code is based on STS standard or talk to SDL if they have ready to use such service which can be leverages.

Part 2
This section is involved in ingesting assets into DAM.

Following are code snippet for digital assets migration/ingestion. code. first section create a program id and upload image

second code snipped create a distribution associated with program id.

 public string UploadFile(long folderId, long assetTypeId, long[] tags, string keywords, string fullPathToFileToUpload)  
     {  
       MediaManager2011Client mediaManagerService = null;  
       try  
       {  
         Logger.Info("Uploading Image");  
         UploadInfoData data = new UploadInfoData();  
         data.DistributionCreation=DistributionCreationOptions.OneDistributionPerItem;  
         data.IsIncludedInGalleries = true;  
         data.MakeAvailableForDownload = true;  
         data.MakeAvailableForWebPublishing = true;  
         data.ProgramCreation = ProgramCreationOptions.OneProgramPerItem;  
         mediaManagerService = new MediaManager2011Client("FederationEndpointHttps");  
         string uploadUrl = mediaManagerService.GetUploadUrl(folderId, assetTypeId, tags, keywords, data);  
         WebClient webClient = new WebClient();  
         var response= webClient.UploadFile(uploadUrl, fullPathToFileToUpload);  
         WebHeaderCollection headers = webClient.ResponseHeaders;  
         string assetIdAsString = headers["AssetId"];  
         string programIdAsString = headers["ProgramId"];  
         Logger.Debug(string.Format("Asset Id:{0} & ProgramId:{1} For Uploaded Image", assetIdAsString, programIdAsString));  
         return programIdAsString;  
       }  
       catch (Exception ex)  
       {  
         Logger.Error(string.Format("Exception caught while uploading image-{0}", ex.Message));  
         return null;  
       }  
       finally  
       {  
         if (mediaManagerService != null)  
         {  
           if (mediaManagerService.State == System.ServiceModel.CommunicationState.Faulted)  
           {  
             mediaManagerService.Abort();  
           }  
           else  
           {  
             mediaManagerService.Close();  
           }  
         }  
       }  
}

  public ItemData CreateNewDistributionData(long programId,long folderId,string distributionName, long outletId, IMediaManager2011 client)  
     {  
       var distributionData = new DistributionData();  
       distributionData.Name = distributionName;  
       distributionData.OutletId = outletId;  
       distributionData.OutletType = OutletClassifier.Image;  
       distributionData.ProgramIds = new[] { programId };  
       distributionData.IsLive = true;  
       distributionData.FolderId = folderId;  
       return client.AddItem(distributionData);  
     }  

Part 3
after images are ingested into Media Manager DAM, those needs to be linked to required components in the Tridion. so for External Content Library has to be used. Installation and configuration of ECL are very well documented on SDL Docs.

ECL Service client - Generating ECL stub is quite a task, it involves getting Media manager id and generate a stub in the CMS which actually create a component in CMS with Tcm-id. 

Stub Uri has following format: "ecl:{publicationId}-mm-{dist-Id}-dist-file"

ECL client is used to create ECL stubs in CMS. Check Detailed Article

Automation of ECL stubs creation

To generate a Stub for the image from external sources using External Content library,  its tricky to generate stub. if you have 100s of images to be imported and then putting those into SDL Tridion using ECL, its very time consuming

so we used following way to generate stub at run-time


Code Snippet:

This code create the ECL compatible URI which will be later used to generate the actual stub

 public string GetUris(string publicationId, CustomImage image)  
 {  
             if (image.d != null)  
             {  
               return string.Format("ecl:{0}-mm-{1}-dist-file", publicationId, image.Id);  
             }  
  }  

Following code is baseclass and instantiate the client to create the stub using following settings

    <endpoint name="EclBinaryEndpoint" address="net.tcp://localhost:2660/ExternalContentLibrary/2012/netTcp" binding="netTcpBinding" bindingConfiguration="EclNetTcpBinding" contract="Tridion.ExternalContentLibrary.Service.Client.ISessionAwareEclService"/>  

 public abstract class TridionEclBase  
   {  
     protected static Client tridionClient { get; set; }  
     public TridionEclBase()  
     {  
       tridionClient = Client.GetClientInstance();  
     }  
   }  

following snippet was used to create the actual stub in the CMS

 public class CreateImageStub : TridionEclBase  
   {  
     private SessionAwareEclServiceClient eclServiceClient;  
     public CreateImageStub()  
       : base()  
     { }  
     public Dictionary<string, string> CreateImageStubByEclUir(List<string> eclUrl)  
     {  
       return eclServiceClient.CreateOrGetStubUris(eclUrl);  
     }  
   }  

it returns key value pair

ECLID as key
TCMID as Value