Help with big inserts

Hello everyone, how are you doing ...

I've a problem in here,
I need to make a 1 000 000 rows insert, and it's taking a lot more than 30
seconds, my problem is that it will be a script that will work in many
servers, I've no idea of how many, the set_time_limit does not work for
many of the servers out there, so I'm in trouble, I've tryed to do a refresh
every 100 inserts, but it doesn't work at all, does anyone have a class,
function or piece of code that will do that ?

Or help me doing one ? Remembering that set_time_limit will not work for me.

If anybody has something that can help me, send to peoplel@hotmail please,
tks a lot

I'm using php + mysql !

Freebird

> I need to make a 1 000 000 rows insert, and it's taking a lot more than
> 30
> seconds, my problem is that it will be a script that will work in many
> servers, I've no idea of how many, the set_time_limit does not work for
> many of the servers out there, so I'm in trouble, I've tryed to do a
> (...)

What about making the insertions on a local MySQL database and then using
PHPMyAdmin to export the data to file, and then using PHPMyAdmin to import
the data from file on the servers? Thats what I did in a similar situation.

- Paul

Freebird wrote:

> I've a problem in here,
> I need to make a 1 000 000 rows insert, and it's taking a lot more than 30
> seconds, my problem is that it will be a script that will work in many
> servers, I've no idea of how many, the set_time_limit does not work for
> many of the servers out there, so I'm in trouble, I've tryed to do a
> refresh every 100 inserts,

eh?

> but it doesn't work at all, does anyone have a
> class, function or piece of code that will do that ?
>
> Or help me doing one ? Remembering that set_time_limit will not work for
> me.
>

It's not going to work (even if you use "insert delayed" which you *did*
read about in the manual didn't you - because nobody so stupid as to post
here without RTFM first???)

> If anybody has something that can help me, send to peoplel@hotmail please,

1) that's not a valid email address
2) that's not how NNTP works

C.

> eh?

?

> It's not going to work (even if you use "insert delayed" which you *did*
> read about in the manual didn't you - because nobody so stupid as to post
> here without RTFM first???)

Of course I did, I even wrote my piece of code !!!


> 1) that's not a valid email address

Of course it's not, all you have to do is add the .COM, aren't you smart
enough to figure that out ? lol

> 2) that's not how NNTP works
Colin, it's obvious in your answer that you just can't help, because you
have no idea about how to do that, just don't worry about me, if you can't
help, don't answer, because your answer was just 100% USELESS.

And here, the intention is to help others and get help on php issues, so if
you can't help, just ignore it, try to find something easyer so you can
help, really, I don't care if you're not happy with my post, if you're mad
because I asked for some help in my mail box, just attach the code to your
post, and it's fine, just don't make me loose my time with your stupidity,
because that's not how NNTP works .....


Freebird

Hey Freebird

I am doing something similer where I am inserting 1 to n rows into
mySQL. Tested it on a few records (10k) and all went well, but when I
ran it on 100k+ things went wrong. It wasn't the 30 sec time limit i
had hit, but the 1mb mySQL sql query limit. To solve it i made an
array of insert statements that were about 900kb long then ran them
all. The whole process takes longer than 30 secs, I have not changed
the time limit as I was not aware of it at the time [not been coding
php for that long].

Hope this may shed some light on the problem?

Cheers
Ice



On Sat, 10 Dec 2005 16:34:41 -0200, "Freebird" <peoplel@hotmail.com>
wrote:

>> eh?
>
>?
>
>> It's not going to work (even if you use "insert delayed" which you *did*
>> read about in the manual didn't you - because nobody so stupid as to post
>> here without RTFM first???)
>
>Of course I did, I even wrote my piece of code !!!
>
>
>> 1) that's not a valid email address
>
>Of course it's not, all you have to do is add the .COM, aren't you smart
>enough to figure that out ? lol
>
>> 2) that's not how NNTP works
>Colin, it's obvious in your answer that you just can't help, because you
>have no idea about how to do that, just don't worry about me, if you can't
>help, don't answer, because your answer was just 100% USELESS.
>
>And here, the intention is to help others and get help on php issues, so if
>you can't help, just ignore it, try to find something easyer so you can
>help, really, I don't care if you're not happy with my post, if you're mad
>because I asked for some help in my mail box, just attach the code to your
>post, and it's fine, just don't make me loose my time with your stupidity,
>because that's not how NNTP works .....
>
>
>Freebird
>
>

> I need to make a 1 000 000 rows insert, and it's taking a lot more than 30
> seconds, my problem is that it will be a script that will work in many
> servers, I've no idea of how many, the set_time_limit does not work for
> many of the servers out there, so I'm in trouble, I've tryed to do a refresh
> every 100 inserts, but it doesn't work at all,

What do you mean by "it doesn't work at all"? Do you get some errors?
How does your code look like? What do you refresh?


Hilarion

Freebird wrote:
> > 2) that's not how NNTP works
> Colin, it's obvious in your answer that you just can't help, because you
> have no idea about how to do that, just don't worry about me, if you can't
> help, don't answer, because your answer was just 100% USELESS.

Colin was just trying to point out that emailing you with the answer
means that the other people in the group won't get the benefit of
seeing the answer. I don't think he meant to imply that you are an
obstreperous little wanker who could do with a good slap. And neither
am I, I expect that you are a really nice person underneath.

BTW

You need to output something and call ob_flush() and flush() every so
often while you are inserting your records in order for the page not to
time out. And you need a spell-checker and to grow up.

Ian

> Colin was just trying to point out that emailing you with the answer
> means that the other people in the group won't get the benefit of
> seeing the answer.

Hey, believe me, that's was not all.

I don't think he meant to imply that you are an
> obstreperous little wanker who could do with a good slap. And neither
> am I, I expect that you are a really nice person underneath.

Believe I am, he was really rude to me.


> BTW
>
> You need to output something and call ob_flush() and flush() every so
> often while you are inserting your records in order for the page not to
> time out. And you need a spell-checker and to grow up.

I'll try something and come back soon.


tks a lot

Freebird