1

Closed

Doesn't work in multiple server set up.

description

Me again :)

So the code doesn't seem to work in our set up, so we have two Orchard instances, I load one with Combinator enabled, I have added ./less to my ignore paths.

The first server will create the links as follows.

<link href="/Media/Combinator/Styles/1489901916-1.css?timestamp=129839682310000000" rel="stylesheet" type="text/css" />
<link href="/Themes/ThemeName/styles/styles.less?timestamp=129839682310000000" rel="stylesheet" type="text/css" />
<link href="/Media/Combinator/Styles/1489901916-3.css?timestamp=129839682330000000" rel="stylesheet" type="text/css" />
<link href="/Media/Combinator/Styles/1489901916-4.css?timestamp=129839682340000000" rel="stylesheet" type="text/css" />

I then load up the second server.

<link href="/Media/Combinator/Styles/1489901916-1.css?timestamp=129839682310000000" rel="stylesheet" type="text/css" />
<link href="https://10.20.2.132/Themes/ThemeName/styles/styles.less" rel="stylesheet" type="text/css" />
<link href="/Media/Combinator/Styles/1489901916-3.css?timestamp=129839682330000000" rel="stylesheet" type="text/css" />
<link href="/Media/Combinator/Styles/1489901916-4.css?timestamp=129839682340000000" rel="stylesheet" type="text/css" />

Where the interal IP Address is the first server I loaded it up from, so as a outside user, this link will never work.

(I also think you don't need the timestamp on ignored resources)
Closed Feb 24 at 4:19 PM by Piedone
Please re-open if present.

comments

Piedone wrote Jun 12, 2012 at 12:52 PM

Good to see you :-). Isn't this the same issue you formerly had when using together with the Cache module? Also, do you have distributed caching implemented? (The built-in one doesn't handle multiple nodes.)

Sarkie wrote Jun 21, 2012 at 1:00 PM

I think it is different to be honest, it seems to work "ok" with caching now. No random transaction errors.

Still using inbuilt caching.

I think this will still work with the caching, just store when the resource was created and when it is expiry in the database?

Load up Server 1.

Minifies all the resources etc.

Load up Server 2

Check's database, makes sure the files still exist, then uses them?

What stuff will be cached, once the files have been created?

Piedone wrote Jun 21, 2012 at 3:07 PM

I'm a bit confused about your message, what exactly are you trying to do?

Theoretically, in a multi-server environment, if DB and Media folder are shared or replicated properly, the following will happen:
1) A page with a specific set of resources is hit on one server. These resources were not yet processed (i.e. they're not saved in the DB), so Combinator will process them and save them to the Media folder, to the DB and also it will saved into the cache (here and subsequently I mean the memcaching ICacheManager) that these set of resources are processed, and what urls the combined resources have.
2) If on any other server the same page is hit, the request is forced to wait until the one on the first server finishes.
3) Now the combined resources are there in the Media folder, their corresponding entries in the DB and the latter also in the cache on the first server.
4) All waiting servers can continue.
5) Regardless if previously it waited or not, all other servers will now see from the DB that the resources are processed. They will independently fetch the data from the DB and independently cache them.

So far all good. What breaks the whole thing is if the cache is emptied. Because caching is not shared (with the default implementation), no other servers will recognize the cache being emptied, and they will continue to serve the now deleted combined resources until the whole cache gets purged (e.g. with an AppDomain restart).

If you try to run Orchard without a proper distributed caching system (again, I'm meaning an ICacheManager implementation here), you'll run into various problems.

Sarkie wrote Jun 21, 2012 at 4:42 PM

Ok, sorry about that! I'll reply and clarify.

1.) The media folder is shared via
\uncpath\media
Both servers point to the same database.

2.) How is it forced to wait? The page might be pushed onto another server so won't the app decide?

3.) Ok, why is the cache needed here? For the list of resource files?

I could create a Signal background Task, so when you do a Signal.Trigger('cache') it writes to the database, and each server, checks the last time it accessed the table and goes through each one on the local server doing the triggers...

But in my situation it should be working, unless I clear the cache on one of the servers?

This still doesn't show why the ip is showing up on the ignored resource (/.less)?

Piedone wrote Jun 21, 2012 at 5:12 PM

Yeah, until the cache is not cleared, it should work on all of the servers (though I've never tried this).
2) With a locking cache manager implementation (that uses an ICacheManager instance but forces the code to wait if there is a concurrent thread writing to the cache entry).
3) Yes.

If you can manage to implement signals to reliably propagate across all the nodes than it should solve the problem, yes.

I can't think of anything about the IP, but I can recall that you had this same issue once, and as far as I can remember, it was some incompatibility with the Cache module.

Sarkie wrote Jun 22, 2012 at 1:33 PM

Hey,

Had some time to look at it today. In the database for an excluded file, it had the full path. Looking in the code.

CombinatorService.cs
Line : 212.
resource.OverrideCombinedUrl(resource.PublicUrl);

The issue here is, its got the resource with the IP of one of the servers. So in my case, it showed up the wrong server. Since it is excluded, I changed the interface for the Method (and implementation and the settings) and changed it from a Uri, to a string. Then I did this.

resource.OverrideCombinedUrl(resource.Resource.Url);

And I also changed line 236 to have a check to stop adding timestamps to overriden urls.
 if(!resource.CombinedUrlIsOverridden)
                    uriBuilder.Query = "timestamp=" + resource.LastUpdatedUtc.ToFileTimeUtc();
I'm not sure if this is the correct way for you, or have something that says "IsExcludedResource" and then do things differently. But I'm just testing now, but I think this will be why.

Sarkie wrote Jun 22, 2012 at 1:58 PM

My code fix, doesn't seem to work with <!--[if lt IE 9]> stuff, but time for lunch.

Piedone wrote Jun 28, 2012 at 7:21 PM

The point of this PublicUrl is that it includes the full url of any resource. resource.Resource.Url not necessarily has that capability, as BasePath could or could not be filled. Actually the domain part of PublicUrl is filled from the request; so I guess in your web farm nodes see the request with not the domain, but their IP included?

The fix for not adding the timestamp to urls with overridden url is fine thanks! I'll include it in the code base.

Sarkie wrote Jun 29, 2012 at 10:26 AM

Hmm, do you need a PublicUrl though? Why not just have the relative url, what are you gaining from having the public url?

The reason it doesn't work in our set up is we have a service that pings the server's ip's to check if they are awake, and can't do that by dns name, so has to do it via IP, I'm looking into it now.

Good good!

Piedone wrote Jul 19, 2012 at 11:01 AM

I can't recall precisely, but this technique was needed so all resources can be stored consistently. I didn't start with a full URL but later discovered it is the way to go.

Piedone wrote Jul 19, 2012 at 12:07 PM

Now I tried it with 1.5 and the timestamp is not added to excluded resources in the output (funny thing I don't know why: it is added in code).

Piedone wrote Jul 19, 2012 at 6:21 PM

I've pushed some changes that could solve this problem as well (and now the timestamp is not added to excluded resources either). Could you please check it out?

Sarkie wrote Jul 20, 2012 at 10:21 AM

Seems to be working :)

I had to edit the code and took out the 1.5 Exception.IsFatal() since we are still on 1.4.2 here... :)

I'll put it on our next white label and see how it works!

I will hopefully be sharing my Distributed Signals Module for the other servers soon :)

Thanks again!!!

Piedone wrote Jul 20, 2012 at 11:46 AM

Awesome! Distributed signals would be fantastic for those on a Web Farm, but without distributed caching.

Piedone wrote Jul 26, 2012 at 10:17 PM

Please notify me whether 1.4 fixed this issue.

Sarkie wrote Jul 27, 2012 at 8:52 AM

It is scheduled to be tested Monday, I couldn't get it into the release for this week, sorry!

Piedone wrote Jul 27, 2012 at 1:07 PM

OK, cool!

Piedone wrote Sep 20, 2012 at 7:28 PM

Any news on this?

Sarkie wrote Sep 21, 2012 at 1:03 PM

No sorry, its not come up again and its too much for testing... I'll let you know as soon as I test it.

But on my box, of multiple app pools, it seems to work perfectly :)

Sign in to add a comment