running a very long process from php

Hi.

For my CMS, i need to do imports of photos and videos.
For uploading large files i have a Java FTP applet with automated resume, so
no problems there.

But: unzipping them and importing them might take a long time if it's a
large batch (1000s of pictures/videos).

Currently i split this up by importing only 1 item per php call from the
browser, to get around the hoster's limitation of 30s php timeout.
But since my hoster wont support video-importing (ffmpeg) anyhow, i have
decided to do some tasks from my home server over which i have total
control.

I'd also like to get rid of the requirement to have a browser open during
the import process.
Not only is it cumbersome for the user, if the internet connection fails
then the import is interrupted too.

I'm considering to (after ftp has completed) call a php script on my
homeserver that calls scripts on the hoster.
Scripts on the hoster would write status-info to a file, which is then read
from the originating browser.
And if a video-file is encountered, my homeserver downloads it, then ffmpegs
the meta-info and thumbnails, uploads those results to the hoster, calls a
'receiver' script on the hoster to put the thumbnails and meta-info in the
database..

Good idea?
Or do you have a better one? :)

Rene Veerman wrote:
> Hi.
>
> For my CMS, i need to do imports of photos and videos.
> For uploading large files i have a Java FTP applet with automated resume, so
> no problems there.
>
> But: unzipping them and importing them might take a long time if it's a
> large batch (1000s of pictures/videos).
>
> Currently i split this up by importing only 1 item per php call from the
> browser, to get around the hoster's limitation of 30s php timeout.
> But since my hoster wont support video-importing (ffmpeg) anyhow, i have
> decided to do some tasks from my home server over which i have total
> control.
>
> I'd also like to get rid of the requirement to have a browser open during
> the import process.
> Not only is it cumbersome for the user, if the internet connection fails
> then the import is interrupted too.
>
> I'm considering to (after ftp has completed) call a php script on my
> homeserver that calls scripts on the hoster.
> Scripts on the hoster would write status-info to a file, which is then read
> from the originating browser.
> And if a video-file is encountered, my homeserver downloads it, then ffmpegs
> the meta-info and thumbnails, uploads those results to the hoster, calls a
> 'receiver' script on the hoster to put the thumbnails and meta-info in the
> database..
>
> Good idea?
> Or do you have a better one? :)
>
sounds like it's time to get your own dedicated server to be honest; in
the time it takes you to do that semi-manual process I'm sure you could
easily do a couple of jobs to fund the server if needs be;

Many Regards

Nathan

Nathan Rixham wrote:

> sounds like it's time to get your own dedicated server to be honest;

Yeah, that was my first thought too.

> in the time it takes you to do that semi-manual process I'm sure you
> could easily do a couple of jobs to fund the server if needs be;

They're not even very expensive - how about EUR50/month for a dual-core
AMD with 2Gb RAM, dual 400GB SATA disks and unlimited bandwidth.


/Per Jessen, Zürich