experchange > programmer

Jeff Rodriguez (12-05-03, 08:32 AM)
Here's what I want do:

Have a main daemon which starts up several threads in a Boss-Queue structure.

From those threads, I want them all to sit and watch a queue. Once an entry
goes into the queue, grab it and run a system command.

Now I want to make sure that system command doesn't hang forever, so I need some
way to kill the command and have the worker thread go back to work waiting for
another queue entry.

Now the reason for the crossposting is because it could be done without threads.
I'm trying to figure out how to go about it, a non-blocking open() somehow? I
just now thought of popen(), but how to do so in a non-blocking way?

Any thoughts?

Jeff
Jack Klein (12-05-03, 08:36 AM)
On Thu, 04 Dec 2003 23:32:41 -0700, Jeff Rodriguez
<newsgroup1> wrote in comp.lang.c:

In the future please leave comp.lang.c out of your cross-post list
when asking platform specific questions, they are off-topic h ere.

> Here's what I want do:
> Have a main daemon which starts up several threads in a Boss-Queue structure.


There are no daemons or threads in the C language.
Joseph Dionne (12-05-03, 04:44 PM)
Here is a real rough example on how to use popen(). I use this
technique quite often.

{
FILE *running;

running = popen("ls -l","r");

if (running)
{
fd_set rFds = {0};
int nFds = -1;

FD_SET(fileno(running),&rFds);
nFds = select(1+fileno(running),&rFds,0,0,0);
switch(nFds)
{
/* decode what happended here */
}
}
}

I'd recommend you use poll() for waiting (select() is easier for the
example). Then you can wait as long as you wish, and closing running
should signal a broken pipe to the process, thus terminating it.

I don't know if you need bi directional interaction with the other
process, but the theory will still be the same, you just need to handle
the "other" handle before the popen() call

Jeff Rodriguez wrote:
[..]
CBFalconer (12-05-03, 07:43 PM)
*** Rude top-posting fixed. Follow-ups set. ***

Joseph Dionne wrote:
[..]
> I don't know if you need bi directional interaction with the other
> process, but the theory will still be the same, you just need to
> handle the "other" handle before the popen() call


This thread is entirely off topic for c.l.c, inasmuch as the C
language knows zip about threads. C.l.c. also does not condone
top-posting.
Bjorn Reese (12-05-03, 08:06 PM)
[ comp.lang.c added back in ]

On Fri, 05 Dec 2003 17:43:53 +0000, CBFalconer wrote:

> This thread is entirely off topic for c.l.c, inasmuch as the C
> language knows zip about threads. C.l.c. also does not condone
> top-posting.


As a reader of comp.unix.programmer, where the thread is
relevant, may I humbly ask you lot on comp.lang.c to practice
what you preach by not cross-posting your replies about the
charter of comp.lang.c, which is off-topic on the other groups?

Thank you.
Joseph Dionne (12-05-03, 08:09 PM)
Well, technically, the 'c' language knows zip about sockets, however we
discuss sockets here.

CBFalconer wrote:
[..]
Kevin Goodsell (12-05-03, 09:40 PM)
Bjorn Reese wrote:
> [ comp.lang.c added back in ]
> On Fri, 05 Dec 2003 17:43:53 +0000, CBFalconer wrote:
> As a reader of comp.unix.programmer, where the thread is
> relevant, may I humbly ask you lot on comp.lang.c to practice
> what you preach by not cross-posting your replies about the
> charter of comp.lang.c, which is off-topic on the other groups?


Sorry, no. That's not a reasonable request. It does no good for us to
tell others in comp.lang.c that a message is off-topic here while people
in other groups merrily cross post more and more off-topic replies to
our group. When somebody cross-posts an off-topic message, we have to
ask the other groups to remove us from the cross-post list when
replying. I expect people in any other group to do the same, and
certainly wouldn't complain if a member of some other group replied to a
cross post in c.l.c to tell us that the thread is not topical in their
group. In fact, it's nice to know, since it removes all doubt about
whether a reply should be cross-posted back to that group or not.

We in comp.lang.c do our best to be good neighbors. We're sorry for the
noise, but blame the original cross-poster, not us.

Disclaimer: I don't speak for the whole group. Any or all of the other
members of c.l.c may disagree with me.

-Kevin
Jeff Rodriguez (12-06-03, 04:22 AM)
Joseph Dionne wrote:
> Well, technically, the 'c' language knows zip about sockets, however we
> discuss sockets here.
> CBFalconer wrote:


I'm sure I can figure out how to kill it if it becomes unruly, but first I need
it's PID. That seems to be the way to go I think, unless anyone else has any
other ideas?

Jeff
Jeff Rodriguez (12-06-03, 04:33 AM)
Jeff Rodriguez wrote:

> Joseph Dionne wrote:
> Ok, say I do use popen. How do I get the PID of my child process?
> I'm sure I can figure out how to kill it if it becomes unruly, but first
> I need it's PID. That seems to be the way to go I think, unless anyone
> else has any other ideas?
> Jeff Actually, I just found this little tidbit:


> Don't use popen() .
> Use fork() and exec*() instead. You will get the pid (fork() return value),
> and it's more flexible and secure than popen() because it doesn't spawn a
> shell, but a real command.


Does anyone have objections to using that method for any reason? I would think
this cause duplicates of much of the memory already in use by the parent
process, is that memory free()'d when I use one of the exec*() functions?

Now what if I wanted to do bi-directional communication with that process... do
the exec*() functions destory all variables (pertains to previous paragraph). If
I didn't want to use a file in the operating system, how would I communicate
between the two?

Jeff
Jeff Rodriguez (12-06-03, 04:36 AM)
Jeff Rodriguez wrote:

[..]
> previous paragraph). If I didn't want to use a file in the operating
> system, how would I communicate between the two?
> Jeff Apologies, this just became off-topic for c.p.t


Thank you for your help.

Jeff
David Schwartz (12-06-03, 05:52 AM)
"Jeff Rodriguez" <newsgroup1> wrote in message
news:0409

> > Use fork() and exec*() instead. You will get the pid (fork() return value),
> > and it's more flexible and secure than popen() because it doesn't spawn

a
> > shell, but a real command.


> Does anyone have objections to using that method for any reason? I would think
> this cause duplicates of much of the memory already in use by the parent
> process, is that memory free()'d when I use one of the exec*() functions?


Fortunately, duplication is nearly free on every modern UNIX. The memory
is not freed because it's still in use by the other copy, its reference
count is reduced though.

> Now what if I wanted to do bi-directional communication with that process... do
> the exec*() functions destory all variables (pertains to previous paragraph). If
> I didn't want to use a file in the operating system, how would I communicate
> between the two?


You would open a pipe before calling 'fork'. After 'fork', the parent
would close one end of the pipe and the child would close the other.
Commonly, the child's pipe end is then 'dup2'd onto stdin, stdout, or
stderr. Then you call 'exec'.

DS
Sean Burke (12-06-03, 07:45 AM)
Jeff Rodriguez <newsgroup1> writes:

> Jeff Rodriguez wrote:
> Actually, I just found this little tidbit:


Well, popen() uses fork and exec too. I would put it a different way,
and say that if the standard popen() doesn't meet your needs, you may
need to code a custom version that does what you want.

> Does anyone have objections to using that method for any reason? I would think
> this cause duplicates of much of the memory already in use by the parent
> process, is that memory free()'d when I use one of the exec*() functions?


Why are you worrying about this? The only process on a Unix system
that isn't created by a fork/exec, is init. popen() uses fork/exec,
and so does system().

> Now what if I wanted to do bi-directional communication with that process... do
> the exec*() functions destory all variables (pertains to previous paragraph). If
> I didn't want to use a file in the operating system, how would I communicate
> between the two?


You need to decide whether bi-directional communication is a requirement.
If it is, then you should first determine whether the standard popen()
will support that. Some OS's (e.g. Solaris, FreeBSD) have pipes that allow
bi-directional communication, and others (e.g. Linux) don't.

If you need bi-directional communication, then you will definitely need to
do a custom popen(), since you will need separate pipes for the child process's
stdin and stdout.

If you wish to avoid threads and instead monitor multiple child processes
using select(), then there is another issue, which is that popen() provides
a buffered IO stream (FILE *) rather than a file descriptor. You will need
the file descriptor for select(). You can get the underlying file descriptor
using fileno(), and you can make it non-blocking for use with select() via
fcntl(). But you need to be very careful if you are mixing buffered IO
(fread, fscanf, etc. ) with select(), since select doesn't know whether
there is data in the buffer.

I would start with standard popen(), and see how far I got with that.
You may find that popen() doesn't provide any advantages compared to
system(), or you may find that you need to customize popen() for your
needs.

-SEan
Barry Margolin (12-06-03, 12:05 PM)
In article <4b5Ab.352$_r6.47>,
Kevin Goodsell <usenet1.spamfree.fusion> wrote:

> We in comp.lang.c do our best to be good neighbors. We're sorry for the
> noise, but blame the original cross-poster, not us.


No you don't, and I don't think you're sorry. I've never seen any other
group be so routinely hostile to innocent newbies who don't know any
better. Their thinking was probably something like "I'm programming on
Unix in C, so I'll ask in the Unix and C groups." Is that *so*
unreasonable that they need to be made to feel like idiots?

You can clearly see that the message is cross-posted to a Unix group.
Then when it makes mention of a Unix function (which you probably knew
was going to happen) you typically give some kind of sarcastic response,
embarassing them as if they're the first one to ever have made this
mistake.

We occasionally get questions in comp.unix.programmer that are about
straight C, with nothing Unix-specific. I've rarely seen anyone here
respond "Sorry, that's not a Unix question, go ask in comp.lang.c." We
answer them, and then perhaps mention your fine newsgroup (as well as
some related alt.learn groups, if it seems appropriate) for future
questions of the same type.

You guys are like a grumpy neighbor. Kids don't want their ball to get
thrown into his yard by accident, because he won't give it back without
a fight.
Barry Margolin (12-06-03, 12:12 PM)
In article <3FD14000.1060409>,
Jeff Rodriguez <newsgroup1> wrote:

> Actually, I just found this little tidbit:
> Does anyone have objections to using that method for any reason? I would
> think
> this cause duplicates of much of the memory already in use by the parent
> process, is that memory free()'d when I use one of the exec*() functions?


exec() completely replaces the process's memory with that of the new
program. Think about it: except for the init process that's created at
boot time, every process on Unix was created by forking and usually
exec'ing something immediately thereafter. Every program run from the
shell was fork/exec'ed by the shell. By the time you get to an
application program, you're at least 4 levels deep in this, and it would
make little sense if each one of them were layered on top of a copy of
the parent rather than replacing it.

> Now what if I wanted to do bi-directional communication with that process...
> do
> the exec*() functions destory all variables (pertains to previous paragraph).
> If
> I didn't want to use a file in the operating system, how would I communicate
> between the two?


Use a pair of pipes, one for parent->child and the other for
child->parent. Some operating systems support bidirectional
communications through a single pipe, but if portability is a concern
you shouldn't depend on this.
Barry Margolin (12-06-03, 12:16 PM)
In article <x7vfoulcsh.fsf>,
Sean Burke <foobar> wrote:

> You need to decide whether bi-directional communication is a requirement.
> If it is, then you should first determine whether the standard popen()
> will support that. Some OS's (e.g. Solaris, FreeBSD) have pipes that allow
> bi-directional communication, and others (e.g. Linux) don't.


Even though the underlying pipes may support bidrectional communication,
popen() returns a stdio stream that's only open in one direction. I
guess you could use fileno() to get the pipe descriptor and bypass
stdio, but I'm not sure how well stdio will deal with it when you
finally call fclose(). Mixing Unix I/O and stdio is not for beginners;
if he can't program fork/exec himself, he's probably not ready to deal
with that.

Similar Threads