This project has been created as part of the 42 curriculum by maaugust.
A Note on This Repository: > I built this project simultaneously with
minitalk, but due to 42's concurrent project limits, I could only submit one for an official grade. I chose to see Pipex through to completion anyway. Not only did it provide immense value in understanding IPC and process management, but it also served as the perfect architectural foundation for the highly complexminishellproject (which I have since completed). While this repository does not have an official Moulinette score, it was built to a 125% bonus standard. It successfully handles unlimited piped commands, in-memoryhere_docrouting without data races, exact bash-equivalent exit codes, and strict FD management.
Pipex is a project in the 42 curriculum that explores UNIX mechanisms in detail by recreating the behavior of shell pipes (|) and redirections (<, >, <<, >>).
The primary goal of this project is to learn how to manipulate file descriptors, handle process creation using fork(), and establish inter-process communication (IPC) using pipe(). The program intercepts commands, routes their standard input and output through anonymous pipes, and executes them concurrently while properly managing memory and zombie processes.
The mandatory program takes exactly four arguments (two files and two commands) and behaves exactly like the following bash command:
$< infile cmd1 | cmd2 > outfile
- Accurately resolves command paths using the environment (
envp). - Fully protects against execution failures, locked files, and missing permissions, returning precise bash-like exit codes (126, 127).
The bonus implementation drastically expands the scope of the program to handle complex, dynamic routing:
- Multiple Pipes: Supports an arbitrary, unlimited number of piped commands:
$< infile cmd1 | cmd2 | cmd3 | ... | cmdn > outfile - Here Document (
<<): Supportshere_docinput, reading from the standard input until a specificLIMITERis found, and appending the final output>>to the target file:$ cmd1 << LIMITER | cmd2 | ... | cmdn >> outfile
Per the core requirements, here is the justification for the architecture used in this project.
To comply with the strict 42 Norm (max 4 parameters per function), state is managed via a central t_data structure. It stores the input/output FDs, the dynamic process ID array, and tracks the active pipes, allowing seamless cleanup in case of a fatal error at any point during execution.
Standard approaches to handling multiple pipes often involve allocating a massive 2D array of file descriptors (int **pipes). This scales poorly and creates nightmare cleanup scenarios.
- The Logic: This project uses a Sliding Window approach. Only one active pipe is created per iteration. After the child process forks and duplicates the necessary FDs, the parent process closes the write-end and saves the read-end into
data->prev_fd. The next command then reads fromprev_fdinstead of a new array index. This keeps the memory footprint infinitely small regardless of the number of commands.
In the bonus part, to mimic here_doc, standard shells usually write the input to a hidden temporary file (e.g., in /tmp/).
- The Problem: Without access to randomizing functions like
mktemp(), hardcoding a temporary file (e.g.,.heredoc_tmp) introduces a severe race condition if two instances of the program run simultaneously. - The Solution: This project utilizes an entirely in-memory anonymous pipe to buffer the
here_docinput. While this restricts the maximum input to the UNIX pipe buffer limit (usually 64KB), it strictly prioritizes data integrity and absolute process isolation, guaranteeing zero residual files if the program crashes.
To compile the project, run the following commands in the root of the repository.
- To build the mandatory executable (
pipex):
make- To build the bonus executable (
pipex_bonus):
make bonusmake clean: Removes the compiled object files (.o).make fclean: Removes object files, the executables, and cleans the embeddedlibft.make re: Performs a clean re-build.
1. Mandatory Part: Pass the infile, two commands, and the outfile.
./pipex infile "ls -l" "wc -l" outfile(Equivalent to: < infile ls -l | wc -l > outfile)
2. Bonus Part (Multiple Pipes): Pass an arbitrary number of commands between the files.
./pipex_bonus infile "grep a" "sort -r" "awk '{print \$1}'" "wc -l" outfile(Equivalent to: < infile grep a | sort -r | awk '{print $1}' | wc -l > outfile)
3. Bonus Part (Here Document):
Use here_doc as the first argument, followed by the limiter keyword.
./pipex_bonus here_doc EOF "grep error" "wc -l" outfile(Equivalent to: grep error << EOF | wc -l >> outfile)
Pipex has many hidden traps regarding process management, memory leaks, and exit codes. Rigorous testing is essential.
When testing absolute paths (e.g., /usr/bin/invalid_cmd), many implementations accidentally overwrite the errno variable during their error-printing phase (like calling free() or write()), resulting in a 126 (Permission denied) exit code instead of the expected 127 (Command not found). Ensure your errno is saved immediately after execve fails to return accurate, bash-like exit codes.
A true test of your bonus multiple-pipe implementation is to pipe hundreds of commands simultaneously (e.g., cat piped 500 times). If your architecture hoards file descriptors, it will crash. The sliding window algorithm ensures only the necessary FDs are open at any given time, preventing EMFILE (Too many open files) errors.
While Pipex handles a linear pipeline, the architectural choice to use a Sliding Window for file descriptors was a deliberate preparation for minishell. In a full shell environment with bonus features, complex user input is parsed via recursive descent into a binary Abstract Syntax Tree (AST). Because pipelines are dynamically generated and deeply nested, you cannot safely rely on fixed 2D arrays to store pipes. By ensuring that only one active pipe is created at a time and the read-end is passed forward via data->prev_fd, this exact execution logic can be ported directly into minishell's recursive AST executor. When traversing a PIPE node, it effortlessly routes the FDs through the left and right branches, guaranteeing zero FD leaks and preventing exhaustion regardless of pipeline complexity.
Because pipex utilizes dynamically allocated arrays for process IDs and environmental path matrices, it is critical to verify memory safety. Furthermore, tracking open File Descriptors is essential, as leaving pipes open will cause the program to hang. Use the --track-fds=yes flag in Valgrind to ensure absolute system safety across both mandatory and bonus executables:
Scenario 1: Mandatory - Invalid Command Leak
Forces the mandatory program to parse the environment paths, fail to find the binary, free the 2D arrays, and exit securely without executing.
valgrind --leak-check=full --show-leak-kinds=all --track-fds=yes ./pipex infile "ls -l" "invalid_command" outfileScenario 2: Bonus - Multiple Pipes & Invalid Command
Checks the bonus logic to ensure the dynamic PID arrays and sliding window pipes don't leak when a middle command fails.
valgrind --leak-check=full --show-leak-kinds=all --track-fds=yes ./pipex_bonus infile "ls -l" "invalid_command" "wc -l" outfileScenario 3: Bonus - here_doc Static Buffer Drain
Ensures the get_next_line static buffers are cleanly freed when the LIMITER is reached and that the anonymous memory pipe is closed correctly.
valgrind --leak-check=full --show-leak-kinds=all --track-fds=yes ./pipex_bonus here_doc EOF "grep a" "wc -l" outfileExpected Valgrind Output for all scenarios: All heap blocks were freed -- no leaks are possible
FILE DESCRIPTORS: 3 open at exit. (These are standard 0, 1, 2)
To ensure your Pipex handles temporary zombie processes, exact exit codes, and memory leaks perfectly, I highly recommend running your code through these community testers:
- michmos / 42_pipex_tester - Excellent for checking exact bash-equivalent exit codes.
- bastienkody / pipex_tester - Great for catching temporary zombie processes and FD leaks.
⚠️ IMPORTANT NOTE FOR BONUS TESTING: > Both of these community testers assume that runningmake bonusoverrides the mandatory./pipexbinary. Because this repository properly isolates the bonus into a separate./pipex_bonusexecutable, running the testers on the bonus part will initially fail (since they will pass 5+ arguments to the mandatory binary).
- The Workaround: Temporarily open the
Makefileand changeB_NAME = pipex_bonustoB_NAME = pipex. Runmake fcleanandmake bonus. The testers will now correctly evaluate your multiple pipes andhere_doclogic. Do not forget to change it back before submitting to Moulinette!
Moulinette relies on a program called norminette to check if your files comply with the 42 Norm. Every single .c and .h file must pass this check. If there is a norm error, you will receive a 0.
The 42 Header:
Before writing any code, every file must start with the standard 42 header. norminette will automatically fail any file missing this specific signature.
/* ************************************************************************** */
/* */
/* ::: :::::::: */
/* pipex.c :+: :+: :+: */
/* +:+ +:+ +:+ */
/* By: maaugust <maaugust@student.42porto.com> +#+ +:+ +#+ */
/* +#+#+#+#+#+ +#+ */
/* Created: 2025/11/21 02:11:40 by maaugust #+# #+# */
/* Updated: 2025/11/25 03:30:20 by maaugust ### ########.fr */
/* */
/* ************************************************************************** */Run the following command in the root of your repository to check all your files at once:
norminette -R CheckForbiddenSourceHeader srcs/ bonus/ includes/System Manuals:
man 2 pipe,man 2 fork,man 2 dup2,man 2 execve- Essential Linux programmer's manuals for understanding process creation, file descriptor duplication, and command execution.
Articles & Guides:
- Pipex Tutorial - 42 Project (Medium): A comprehensive guide on how the pipex architecture works under the hood.
- Pipex: Understanding Pipelines in C (Medium): A great article explaining the core concepts and file descriptor routing.
- Pipex 42 (Medium): A detailed walkthrough explaining the transition from shell commands to C process management.
Video Tutorials:
- CodeVault - Unix Processes in C: An absolute goldmine playlist for understanding forks, pipes, and wait functions.
42 Standards:
- 42 Norm V4: The strict coding standard for 42 C projects.
- Official 42 Norminette Repository: The open-source linter enforcing the strict 42 coding standard.
In the spirit of transparency and the learning objectives of the 42 curriculum, here is how AI tools were utilized during this project:
- Conceptual Validation: Used as a sounding board to verify architectural trade-offs, such as analyzing the race conditions of temporary files versus the UNIX buffer limits of anonymous pipes for the
here_docimplementation. - Documentation & Formatting: Assisted in drafting comprehensive Doxygen comments for the source files, reviewing general code modularity, and structuring this
README.mdlayout. - Zero Code Generation: No core logic was generated by AI. All file descriptor management, process forking, inter-process communication, wait loops, and execution routing were 100% manually coded. This strict boundary was maintained to ensure a fundamental, hands-on mastery of UNIX system calls and IPC.